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EXECUTIVE  SUMMARY 


This  study  is  an  initial  attempt  to  investigate  the  emerging  net-centric  warfare 
concept  from  the  perspective  of  the  user  interface  and  team  performance.  Net-centric 
warfare  is  enabled  by  advanced  Information  technology  (IT)  configured  as  an  enterprise 
system  of  systems.  The  enterprise  system  concept  supports  the  inclusion  of  a  diverse  set 
of  databases  and  application  software  that  all  contributed  to  the  operational  work 
processes  contained  in  a  battlespace  management,  command,  control,  computers,  ^d 
intelligence  (BMC4^  system.  Because  this  form  of  enterprise  system  includes  a  wide 
variety  of  applications  produced  by  different  vendors,  it  raises  new  challenges  for  user 
interface  design  to  support  coordinated  and  distributed  teamwork:  Do  we  need  a  new 
conceptualization  of  the  user  interface  to  support  distributed  collaborative  work?  How 
can  the  interface  itself  help  provide  a  common  work  focus  when  many  different 
applications  must  be  used  in  work?  How  does  the  user  interface  relate  to  distributed 
teamwork?  This  study  addresses  these  questions. 

The  report  lays  out  a  conceptual  framework  as  the  basis  for  establishing  a 
scientific  foundation  for  user  interface  design  in  the  enterprise  system  context.  This 
framework  includes  a  joint  functional  layer  added  to  the  information  technology  (IT) 
infrastructure.  Design  requirements  for  this  layer  of  the  collaborative  interface  are 
derived  from  principles  of  Cognitive  Systems  Engineering  and  principles  derived  from 
the  study  of  human  expertise.  The  conceptual  framework  for  the  user  interface  is 
described  in  detail. 

In  conjunction  with  the  theoretical  development,  an  experimental  tested  was 
created  to  empirically  investigate  user  interface  issues  associated  with  distributed 
teamwork  in  an  enterprise  IT  context.  An  important  characteristic  of  the  testbed  is  that  it 
includes  a  heterogeneous  array  of  computer  platforms  and  commercial  off-the-shelf 
(COTS)  products.  In  this  study,  the  testbed  was  used  to  host  a  BMC4I  scenario  that 
required  joint  collaborative  work  across  Air  Force  and  Army  umts.  This  military  scenario 
provided  an  additional  means  to  uncover  other  issues  important  to  user  interface  design  in 
an  enterprise  IT  environment. 

The  report  summarizes  the  conceptual  framework  for  collaborative  user  interface 
design,  a  description  of  the  enterprise  system  research  testbed,  and  a  discussion  of  the 
lessons  learned  from  the  informal  experimentation  of  distributed  teamwork  using  the 
BMC4I  scenario. 
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I. 

INTRODUCTION: 

TEAMWORK  THROUGH  EMERGING  INFORMATION 

TECHNOLOGY 

Teamwork  and  coordinated  operations  have  always  been  significant  factors  in  successful 
military  operations.  Recent  experiences  from  operations  in  Desert  Storm  and 
subsequent  large-scale  military  exercises  have  provided  early  demonstrations  of  how 
advances  in  technology  make  it  possible  to  achieve  unprecedented  levels  of  teamwork, 
including  a  more  dynamic  and  fluid  form  of  coordination  and  cooperation.  We  are 
entering  an  era  where  the  extent  of  connectivity  of  people,  information,  tools,  and 
imagery  extends  the  concept  of  a  battlefield  to  include  remotely  located  specialist  and 
commanders,  including  the  Commander-in-Chief  even  when  he  is  in  the  White  House. 
The  Commander-in-Chief,  for  example,  now  has  the  ability  to  interact  directly  with  a 
specific  pilot  in  a  mission,  observing  in  near  real-time  many  aspects  of  the  local  situation 
fi'om  a  single  warfighter’s  perspective.  This  capability  is  being  made  possible  in  large 
part  by  emerging  advances  in  information  technology  (TT).  Some  believe  that  the  changes 
made  possible  by  this  enabling  technology  will  have  a  profound  effect  on  future  warfare. 

Increased  coimectivity  in  both  form  and  quantity  makes  it  possible  for  the  military  to 
consider  highly  interactive,  distributed  organizational  structures.  New  working 
organizations  can  be  created  quickly  to  meet  evolving  contingencies.  Specialists  residing 
in  the  USA  maybe  active  members  of  various  teams  in  a  theater-based  Air  Operations 
Center  (AOC).  Or  an  AOC  may  be  “virtual,”  with  all  its  staff  geographically  distributed, 
thus  providing  an  extended  version  of  the  emerging  concept  of  “reachback.”  Some  are 
even  contemplating  the  possibility  of  dynamically  reconfiguring  joint  operations  while 
actors  are  simultaneously  being  engaged  in  the  execution  of  operational  activities.  A  vast 
array  of  possibilities  for  how  to  improve  fast-paced,  coordinated  military  operations  is 
being  contemplated.  All  of  them  have  implications  for  organization  structure,  team 
processes,  and  human  integration  with  information  technology. 

Operational  concepts  for  warfare  in  the  future  are  not  settled.  There  are  many 
possibilities,  and  we  can  expect  them  to  change  more-or-less  continuously  over  time. 
Indeed,  IT  itself  even  makes  it  possible  to  change  an  operational  concept  more  completely 
and  faster  than  ever  before.  However,  for  initial  planning  purposes.  Joint  Vision  2010 
offers  a  top  level  view  of  the  emerging  military  operational  concepts.  It  outlines  four 
concepts:  (1)  dominant  maneuver,  (2)  precision  engagement,  (3)  focused  logistics,  and  (4) 
full-dimensional  protection.  IT  is  a  critical  enabling  technology  needed  to  realize  these 
concepts  in  practice.  Practical  considerations  that  must  also  be  taken  into  account  include 
leaner  DoD  budgets,  reduced  staffing,  and  a  desire  to  minimize  the  “foot  print”  of  our 
military  forces  in  the  vicinity  of  the  physical  engagement  region. 
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While  IT  is  expected  to  make  new  ways  of  operating  possible,  it  brings  with  it  the  need  to 
develop  a  deeper  and  in  some  cases  new  understanding  of  the  requirements  for  inter  and 
intra  team  organization  and  coordination,  as  well  as  the  need  for  new  requirements  for 
how  best  to  integrate  humans  with  the  IT  media  and  embedded  tools.  Organizational 
processes  supporting  team  and  collaborative  work  are  interactive  with  IT.  The  most 
visible  point  of  interaction  is  at  the  user  intkface. 

This  final  report  covers  work  being  accomplished  under  the  New  World  Vista  initiative  in 
the  area  of  human  interfaces.  It  describes  the  development  of  a  research  program  we  have 
undertaken  to  investigate  user  interface  requirements  for  an  information  technology 
system,  in  terms  directly  related  to  interface  consequences  on  individual  and  teamwork 
processes  and  outcome  performance.  The  purpose  of  this  research  is  to  provide  a 
scientific  foundation  for  establishing  design  requirements  for  ihs  joint  functional 
interface  through  which  te^s  engage  their  IT  support  in  a  highly  distributed  and 
interconnected  workspace.  This  work  is  conducted  under  the  aegis  of  the  Collaborative 
Systems  Technology  Laboratory  (CSTL),  Armstrong  Laboratory  (AFRL/CFHI). 

Section  n  will  provide  a  detailed  discussion  of  the  conceptual  infi-astructure  activities 
aimed  at  building  the  theoretical  foundations  for  ongoing  CSTL  research.  This  will 
provide  a  summary  of  relevant  work,  a  synopsis  of  critical  issues,  and  an  appraisal  of  how 
CSTL  can  fiirther  the  state  of  the  theoretical  art.  Section  m  will  similarly  discuss  the  last 
6  months’  CSTL  physical  infi’astructure  ramp-up  and  introduce  an  initial  simulation 
exercise  illustrating  how  IT  constraints  affect  team  performance  in  a  simulated  BMC4I 
scenario. 
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II. 

PROJECT  DEVELOPMENT: 

CONCEPTUAL  INFRASTRUCTURE  ACTIVITIES 

The  goal  of  this  research  is  to  establish  a  scientific  foundation  for  the  development  of 
design  principles  and  information  requirements  applicable  to  the  team  interface  problems 
related  to  the  effective  exploitation  of  information  technology  (IT)  in  military  operations. 
To  achieve  this  goal  we  have  developed  an  empirically  based  research  program  that  is 
capable  of  deriving  design  principles  from  theories  of  individual  and  team  expertise. 
Design  principles  emerge  based  on  a  mapping  from  performance  theory  to  performance 
modeling  to  principles  of  representation  for  the  individual/team  interface  with  IT. 

In  Section  A,  we  shall  identify  the  problems  underlying  theoretical  deficiencies  in 
addressing  interface  issues  to  date.  Section  B  will  address  broad  CSTL  methodology  by 
specifying  the  process  orientation  of  our  research  agenda  and  introducing  a  model  for 
progressively  refining  interface  concepts  and  applications.  Section  C  will  present  an 
initial  CSTL  interface  model  which  maps  the  functional  interrelationships  among  users 
and  technologies.  Section  D  will  specify  key  features  for  a  model  of  team  performance, 
as  well  as  explaining  how  CSTL  will  adopt  a  skills  perspective  in  analyzing  team  IT 
issues.  Section  E  will  continue  this  performance-oriented  development  by  characterizing 
expertise  as  the  focal  concept  in  such  a  skills  perspective.  Finally,  in  Section  F,  we  shall 
introduce  a  preliminary  workspace  model  drawing  on  the  theoretical  elements  already 
introduced.  This  workspace  model  provides  a  tractable  framework  for  linking  individual 
and  team  performance  in  a  manner  amenable  to  empirical  research. 


A.  Problem  Statement 

Our  research  goals  entail  attention  to  two  critical  themes:  a)  the  utility  of  IT  support  for 
teams  operating  as  coherent  and  effective  units  (as  opposed  to  just  any  collection  of 
individuals);  and  b)  the  unavoidable  dependency  of  task  performance  evaluation  on 
characteristics  of  the  work  situation  and  activities  for  which  such  performance  is 
assessed.  In  fiiis  section,  we  shall  outline  what  we  see  as  the  problematical  state  of  the  art 
in  interface  research  and  development,  and  we  shall  frame  the  perceived  problems  with 
regard  to  these  themes. 

Most  generally,  we  can  state  that  the  working  knowledge  required  to  effect  good 
interfaces  is  not  yet  in  hand.  Regardless  of  the  mass  of  literature  dedicated  to  interface 
design  and  usability,  there  is  still  no  universal  or  uniform  theoretical  foundation  from 
which  we  can  readily  draw.  Although  there  are  numerous  “tips”  and  “recommendations 
derived  from  experiment  and  experience,  these  do  not  sum  up  to  a  cohesive  practical 
substitute  for  such  theory.  Even  taken  in  relative  isolation,  these  practical  results  are  not 
guaranteed  to  be  universally  applicable.  It  has  often  proven  difficult  to  consistently  apply 
the  available  representation  principles  for  single  user  interfaces  due  to  conflicts  when  a 
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designer  tries  to  aehieve  a  set  of  information  delivery  goals.  In  some  instances,  published 
guidelines  that  express  representation  principles  offer  conflicting  positions  (e.g.  Woodson 
&  Conover,  1964,  on  the  use  of  color).  These  problems  exist  in  spite  of  the  fact  that 
many  of  the  design  principles  are  supported  by  a  history  of  somid  scientific  research  on 
human  functional  capacities  (e.g.  Wickens  &  Carswell,  1995;  Teichner  &  Krebs,  1974). 

There  is  a  corresponding  immaturity  in  this  area  with  respect  to  our  critical  themes. 
Recent  reviews  of  interface  design  principles  and  their  foundations  (e.g.,  Bennett  & 

Flach,  1992;  Bennett,  Nagy,  &  Flach,  1997)  confirm  that  work  to  date  does  not  provide 
soimd  design  principles  for  interfaces  specifically  geared  to  support  teamwork  -- 
especially  those  interfaces  implemented  for  the  sort  of  spatially  distributed  organizational 
structures  which  future  military  operations  will  require.  Recent  efforts  aimed  at  deriving 
interface  design  principles  have  shown  increased  awareness  of  the  need  to  relate  human 
processing  to  task  factors.  The  problems  in  addressing  this  need  from  a  traditional 
information-processing  frameworic  will  be  discussed  in  more  detail  below.  The  more 
pragmatic  (and  less  problematical)  body  of  work  labeled  cognitive  engineering  or 
cognitive  systems  engineering  (e.g.  Rasmussen,  1986;  Rasmussen  et  al,  1994;  Flach  & 
Dominguez,  1995;  Woods,  1991;  Woods  &  Roth,  1988)  offers  us  a  more  task-sensitive 
framework  for  pursuing  our  obj ectives. 

Typical  approaches  to  the  development  of  principles  for  interface  design  have  conceived 
of  the  interface  as  consisting  of  an  information  channel  and  a  control  channel.  This  is 
well  illustrated  by  Norman’s  (1986)  “bridges”  model  for  two-way  interaction  between 
user  and  computer.  Such  a  characterization  prioritizes  information  and  information 
processing,  resulting  in  a  strong  linkage  between  interface  research  and  analyses  of  the 
sensory,  perceptual,  and  cognitive  capabilities  of  humans.  Even  within  this  mainstream 
of  interface  research  and  development,  specific  principles  of  information  representation  to 
support  teamwork  have  only  been  sketchily  addressed. 

Based  on  our  analysis,  we  believe  that  current  interface  design  principles  are  deficient  for 
specifying  “good  practice”  in  generating  and  evaluating  mechanisms  and  procedures 
enabling  teams  (as  teams)  to  utilize  information  technologies.  A  primary  reason  for  this 
deficiency  is  a  demonstrable  bias  toward  addressing  IT  usage  solely  in  terms  of 
individuals  (as  opposed  to  teams  or  other  collectives).  In  turn,  this  individual  bias  can  be 
traced  to  two  operant  historical  factors  —  one  involving  theoretical  stance  and  the  other 
involving  the  perspective  adopted  in  assessing  interaction  between  human  and  machine. 
Let  us  now  introduce  these  factors  in  turn. 


1.  Limitations  of  an  information  processing  perspective 

The  first  such  factor  (theoretical  stance)  is  the  cognitivistic  /  symbol-oriented  / 
information  processing  paradigm  which  for  over  three  decades  has  dominated  research 
labeled  as  (e.g.)  artificial  intelligence  (AI),  cognitive  psychology,  and  cognitive  science, 
This  perspective’s  default  scope  of  reference  is  “cognition”,  viewed  as  an  input-output 
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symbolic  process.  As  a  result,  this  orientation  necessarily  frames  phenomena  with 
primary  respect  to  information  resources  and  data  streams  whose  nexus  is  the  individual. 
The  “work”  or  the  “task”  typically  is  addressed  somewhat  one-dimensionally  in  terms  of 
such  symbolic  processing.  Additionally,  the  ascription  of  symbolic  processing  to  the 
personal  faculties  of  a  given  human  lends  this  ^proach  a  necessarily  individual  focus. 

The  extension  of  this  individualistic  orientation  to  information  processing  in  groups  or 
teams  has  been  attempted,  but  the  results  to  date  typically  end  with  vague  allusions  to 
“distributed  cognition”  or  “group  cognition.”  Apparently  the  assumption  is  that 
principles  based  on  human  information  processing  apply  equally  well  to  teams  and 
individuals.  Such  allusions  are  admittedly  enticing,  but  their  utility  is  still  questionable. 
For  one  thing,  simplistic  application  of  an  information  processing  perspective  to  teams 
fails  to  escape  the  constraint  that  it  can  only  address  task  performance  to  the  extent  salient 
factors  can  be  expressed  in  terms  of  data  streams  and  symbolic  manipulations. 
Furthermore,  because  such  terminology  is  illuminating  only  to  the  extent  it  expresses  the 
manner  in  which  a  team  operates  as  a  whole,  it  is  potentially  blind  to  interactivity  among 
team  members.  Finally,  this  sort  of  nomenclature  has  not  yet  achieved  any  uniformity  of 
definition  or  connotation  among  those  who  invoke  it. 

Even  if  a  cohesive  theory  of  “group  cognition”  were  available,  its  information  processing 
orientation  would  still  be  problematical  for  our  research  goals.  Informational  parameters 
determine  task  performance  only  to  the  extent  that  the  task  is  comprehensively  delineated 
in  terms  of  processing  information.  This  means  that  such  a  perspective  can  fall  short  in 
addressing  such  issues  as  team  coordination,  joint  work,  and  work  sharing.  For  example, 
factors  such  as  contexmal  dependencies  and  operational  biases  fall  outside  the  scope  of 
such  an  approach  unless  they  can  be  somehow  cataloged  and  coded  as  processable  data 
themselves.  Thus,  even  if  one  were  to  believe  that  all  salient  task  performance  factors 
could  be  symbolized,  he  /  she  would  still  face  the  prospect  of  an  endlessly  receding 
horizon  of  data  which  must  be  subsumed  to  model  and  manipulate  a  given  work  domain. 
Furthermore,  some  factors  of  demonstrable  salience  to  task  performance  (e.g.,  fatigue, 
stress)  lie  entirely  outside  the  scope  of  an  information  processing  perspective.  Finally, 
this  perspective  is  effectively  blind  to  the  instrumental  (e.g.,  physical)  aspects  of  task 
performance,  leading  to  a  corresponding  blmdness  to  the  “mechanics”  of  collaborative 
activity. 

In  summary,  the  information  processing  perspective  is  derived  from  a  science  of  humans, 
where  humans  are  considered  individually  and  only  as  symbol  processors.  We  are  not 
refuting  the  utility  of  this  perspective;  we  are  only  noting  its  limitations  for  our  stated 
purposes.  Most  important  for  research  methodology  is  the  fact  that  an  information 
processing  perspective  is  limited  m  addressing  either  individual  or  team  task 
performance,  because  it  fails  to  adequately  link  general  behavioral  characteristics  with 
salient  details  of  the  work  domain  and  task.  The  gains  derived  from  a  scientific  analysis 
of  human  capabilities  are  offset  by  a  loss  of  cormectivity  to  the  work  domam  semantics 
that  significantly  influence  work  practices.  This  is  well  demonstrated  by  the  empirical 
work  of  Klem  and  others  refuting  the  notion  of  skilled  performance  as  rational  /  symbolic 
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processing  in  support  of  decision  making  (e.g.,  Klein,  1989;  Klein  et  al,  1986;  Klein  et 
al,  1993).  The  information-processing  framework  is  not  “large  enough”  to  offer  an 
adequate  scientific  foundation  for  principles  of  interface  design.  While  this  approach  has 
proven  useful,  it  does  not  constitute  what  we  believe  to  be  the  optimal  theoretical  stance 
for  our  purposes  —  a  system  science  concentrating  on  the  interaction  of  human  with 
machine  in  the  course  of  doing  work. 


2.  The  limitations  of  interface  research’s  focus  on  the  individual 

The  second  such  factor  relates  to  the  mode  of  conventional  IT  deployment  in  work 
settings  —  each  user  employing  a  desktop  workstation  to  accomplish  his  /  her  individual 
tasks  and  /  or  (via  LAN  of  WAN  connections)  those  tasks  in  which  he  /  she  must 
collaborate  electronically  with  other  team  members.  There  should  be  little  surprise  at  the 
fact  this  has  biased  the  relevant  research  toward  what  the  individual  team  member 
confronts  at  his  /  her  personal  zone  of  contact  with  IT  support  (as  opposed  to  what  the 
team  as  a  whole  must  confront  with  respect  to  its  overall  IT  support  infrastructure).  As  a 
result,  experimental  work  on  human-computer  interaction  (HCI)  has  xmdervalued  both 
team  (as  opposed  to  individual)  performance  and  the  task  context  within  which  that 
performance  is  realized  (Bannpn,  1991). 

As  a  result,  little  or  no  tangible  progress  has  been  made  with  regard  to  mechanisms, 
protocols,  and  representations  explicitly  geared  to  support  team  performance.  The  extent 
of  results  in  this  direction  is  typified  by  Hewitt  and  Gilbert’s  (1993)  general  observation 
that  interfaces  for  team  IT  usage  must  contend  with  three  factors  beyond  those  addressed 
for  individuals’  workstation  interfaces:  a)  communication  among  team  members;  b) 
managing  inputs  from  multiple  users;  and  c)  social  interactivity  among  team  members. 
Fluthermore,  what  little  consideration  has  been  given  team  IT  support  can  be 
characterized  as  focusing  outside  the  scope  of  our  concern.  For  example,  Malone  (1985) 
outlined  a  need  to  address  “organizational  interfaces”,  defined  as  "the  parts  of  a  computer 
system  that  connect  human  users  to  each  other  and  to  the  capabilities  provided  by 
computers."  (1985,  p.  66)  This  definition  would  seem  to  be  relevant  to  our  concerns,  but 
the  fact  is  that  Malone’s  elaboration  of  the  concept  remained  anchored  at  an 
organizational  (i.e.,  enterprise-wide)  scope.  As  such,  Malone’s  models  for  such 
interfaces  were  framed  at  a  level  of  generality  too  broad  to  constructively  inform  us  on 
specific  teams’  IT  requirements. 


B.  A  Theory-Based  Approach  to  Interface  Design  Principles 

What  is  needed  is  a  method  to  derive  interface  design  principles  that  combines 
knowledge  about  human  capabilities  with  knowledge  about  connections  to  both 
individual  and  team  work  processes  as  they  are  operant  in  an  actual  work  domain.  This 
presents  a  substantial  challenge,  since  the  aim  of  science  is  to  produce  explanations  for 
fundamental  phenomena.  Such  phenomena  extend  beyond  the  limits  of  the  idiosyncrasies 
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of  any  specific  task  situation  or  work  domain.  Since  work  processes  in  a  domain  involve 
local  details  of  the  domain,  how  then  can  we  establish  a  scientific  basis  for  design 
principles  if  work  processes  are  important  to  interface  design?  Our  solution  to  this 
problem  is  to  appeal  to  and  extend  the  construct  of  expertise.  Humans  who  are 
considered  to  be  good  at  solving  problems  are  said  to  be  experts.  In  other  words,  they 
have  the  property  of  expertise,  either  God  given  or  acquired.  As  used  here,  expertise  is 
taken  to  be  the  outcome  of  a  skilled  acquisition  process.  Importantly,  it  depends  on 
experience  in  a  work  domain.  Now,  if  we  take  a  process  view  of  expertise,  there  is 
evidence  to  suggest  that  the  processes  used  by  experts  apply  across  domains.  This  is  true 
even  though  specific  domain  content  serves  to  trigger  process  actions. 

THEORY  MODEL  ZONE  OF  PRODUCTS 

RESEARCH 


Figure  n.l:  The  CSTL  Theory-Based  Approach  to  Work-Centered  Interface 

Development 

A  process  model  of  expertise  can  be  used  to  establish  principles  for  interface  design 
based  on  the  assertion  tiiat  the  interface  should  strive  to  support  expert  process  behavior. 
A  process  model  of  expertise  is  general.  It  applies  to  all  experts  and,  therefore;  it  is 
amenable  to  scientific  study  and  can  provide  a  foundation  for  development  of  interface 
design  principles.  Effective  interface  design,  however,  also  depends  on  knowing  the 
“triggers”  embedded  in  actual  domain  situations  that  inust  be  incorporated  into  the  work 
process  if  the  user  interface  to  a  work  system  is  to  be  effective.  Therefore  expertise 
theory,  expressed  as  a  model,  must  be  supplemented  with  a  work  domain  model  that 
captures  such  triggers  before  the  theory  can  support  predictions  of  performance  in  a 
specific  work  domain.  It  is  in  this  way  that  local  domain  semantics  and  global  work 
syntactics  are  combined  in  the  scientific  study  of  expertise.  It  is  for  this  reason  that  a 
theory  of  expertise  is  the  cornerstone  to  a  scientific  approach  to  forming  interface  design 
principles  that  are  sensitive  to  domain  specific  factors. 
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Figure  H.  1  is  a  flow  diagram  of  our  theory-based  approach  to  produce  work-centered 
interface  design  principles  applicable  to  IT.  Twin  process  models  of  expertise  (one  each 
for  individuals  and  teams)  capture  or  leverage  the  related  theories  of  individual  and  team 
expertise.  Both  are  needed  since  an  interface  must  support  individual  work  at  die  same 
time  it  supports  teamwork.  The  theories  are  expanded  into  workspace  models  that 
characterize  the  activity  space  in  terms  of  a  set  of  abstract  constructs.  These  models  are 
“open”  in  the  sense  that  they  support  many  process  trajectories  through  the  activity  space. 
A  more  detailed  discussion  of  theories  of  expertise  and  the  workspace  model  is  presented 
later  in  Sections  E  and  F.  Actor  perceptions  and  events  within  a  work  domain  (IT-Based 
Work  Enviromnent)  trigger  both  situation  understanding  and  process  activities.  Interface 
concepts  derived  from  analyses  of  these  processes,  perceptions,  and  events  will  support 
teamwork  and  individual  taskwork.  These  theory-based  concepts  are  tested  empirically 
through  the  use  of  simulated  work  environments  that  have  characteristics  of  real-world 
military  task  situations.  Importantly,  both  process  data  and  outcome  data  analyses  are 
performed  to  assess  predictions. 

This  model  was  explicitly  crafted  to  allow  for  CSTL  learning  (in  the  sense  of  evolving  the 
tools  employed)  as  well  as  the  export  of  practical  research  products.  Results  of  the 
investigations  are  applied  to  improve  the  theory  and  workspace  models  and  to  produce  a 
stream  of  interface  design  principles  and  information  requirements  for  IT  systems.  As 
implied  by  the  feedback  loops,  the  development  iteratively  modifies  both  theories  and 
models  as  necessary  until  a  mature  state  is  reached. 


C.  Emerging  Model  of  the  User  Interface  for  Highly  Connected  Work 
Environments 

It  is  common  to  regard  the  user  interface  as  an  information  channel  comiecting  a  human 
with  a  machine,  or  connecting  sets  of  humans  through  a  machine  or  medium.  Advances 
in  software  technology  suggest  that  an  information  channel  view  is  no  longer  adequate  as 
the  basis  for  the  design  of  user-system  interfaces.  The  notion  of  an  interface  continues  to 
need  redefinition.  What  is  a  team  interface?  Do  we  gain  any  design  leverage  by  using 
such  a  concept?  What  is  the  interface  to  emerging  IT?  We  have  addressed  the  basic 
question  (What  is  an  interface?)  elsewhere  (Eggleston,  1988).  For  this  research  project,  it 
is  important  to  have  a  clear  understanding  of  the  user  interface  in  an  IT  environment.  We 
suggest  that  FT  may  profoundly  change  the  workspace  and  thus  we  need  to  re-evaluate  its 
functional  piupose  in  a  more  highly  connected,  distributed  work  environment.  The  IT 
interface  needs  to  be  characterized  both  from  a  hardware/software  perspective  and  from  a 
functional/purposive  perspective.  This  section  provides  an  introduction  to  this  topic. 

As  stated  earlier,  IT  offers  an  unprecedented  level  of  connectivity  of  people,  information, 
tools,  and  imagery.  It  is  a  medium  for  work  and  for  exchange.  This  implies  that 
functionally  the  interface  is  more  than  an  information  channel.  It  is  also  a  richly  textured 
work  place  for  people  working  alone  and  in  teams.  To  understand  the  interface 
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IT  provides  a  core  technology  for  the  development  of  higjily  connected,  collaborative 
work  environments.  Figure  n.2  presents  our  model  view  of  a  networked  collaborative 
system.  This  depiction  emphasizes  the  IT  infrastructure  mid  shows  a  progression  from 
specific  hardware  (the  lowest  “layer”)  to  the  software  applications  supporting 
collaborative  tasks  in  network  environments  (“groupware”).  The  vertical  ordering  of 
these  layers  is  not  intended  to  connote  some  strict  hierarchical  segregation.  Instead,  it 
laj^  out  the  relative  dependency  of  elements  required  to  build  up  and  exploit  networked 
information  technologies.  The  lowermost  three  layers  comprise  the  collection  of 
elements  required  to  realize  a  single-user  workstation  (hardware,  operating  system,  and 
application  software).  Individual  users  engage  this  subset  of  the  network  topology  in  any 
case,  and  the  “interface”  elements  shown  represent  the  conventional  usage  of  the  term. 
The  upper  three  layers  represent  the  collection  of  elements  required  to  “enhance”  an 
individual  workstation  to  interoperate  with  others  via  a  network. 
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Figure  n.2  also  shows  the  collection  of  interfaces  in  this  highly  connected  work 
environment.  Participants  in  collaborative  work  effectively  engage  the  technology  and 
each  other  through  three  distinguishable  interfaces.  The  first  (denoted  “interface”  in 
Figure  n.2)  is  the  conventional  zone  of  interaction  between  the  individual  user  and  his  / 
her  workstation.  The  second  (which  we  term  the  joint  junctional  interface)  denotes  those 
work  support  mechanisms  through  which  collaborators  or  team  members  a)  engage  the 
objects  of  their  joint  effort  and  b)  engage  each  other  to  (e.g.)  exchange  and  clarify 
“messages”  to  fkcilitate  coordination,  synchronization,  or  specific  aspects  of  individual  or 
joint  task  work. 

The  reason  for  distinguishing  between  these  types  of  interfaces  is  that  analysis  of  joint 
network  operations  requires  some  means  for  disentangling  the  confusing  overlap  and 
interdependencies  among  each  operator’s  access  to  and  interaction  with  their  immediate 
toolset  (workstation),  their  joint^virtual  product  (shared  information  artifacts),  and  each 
other.  This  preliminary  model  view  accomplishes  this  end.  Moreover,  it  helps  to  clarify 
what  is  missing  fi-om  current  IT  developments  which  are  of  greatest  importance  to  the 
development  of  new  organizational  processes  and  team  performance  within  them. 
Specifically,  little  or  no  attention  has  been  devoted  to  identifying  or  developing  a  joint 
functional  interface.  The  model  raises  this  issue  as  a  central  concern. 


D.  Team  Performance 

The  purpose  of  the  joint  functional  interface  is  to  facilitate  effective  team  performance. 

To  insure  sovmd  investigation  of  the  relation  of  mechanisms  of  such  an  interface  with 
team  performance,  we  need  a  fi-amework  that  can  be  used  to  make  meaningful 
distinctions  among  types  of  teams  and  the  locus  of  work  being  moderated  by  the  IT 
interfaces.  Investigators  who  have  been  researching  different  aspects  of  team  behavior 
have  developed  various  taxonomies  and  theoretical  models  to  account  for  their  findings. 
Recently,  Regian  and  Elliott  (1997)  have  advanced  an  interesting  fi-amework  for 
quantitatively  modeling  the  effectiveness  of  team  performance.  Their  fi-amework  consists 
of  a  clear  definition  of  teamwork  (at  the  functional  piupose  level),  a  taxonomy  that  can  be 
used  to  establish  team  types  in  accordance  with  the  teamwork  definition,  and  a  six-factor 
model  of  team  effectiveness.  It  is  a  comprehensive  fi-amework  for  guiding  team 
performance  research.  It  is  summarized  here  to  show  where  in  this  broad  fi-amework  our 
research  fits  in,  as  well  as  how  our  approach  differs,  in  part  due  to  the  concern  for  the 
interaction  of  teamwork  with  IT. 

Based  on  a  more  or  less  standard  view  of  teams,  as  distinct  fi-om  groups,  Regian  and 
Elliott  derive  a  functional  definition  of  teamwork:  “...team  work  is  the  effective 
managing  of  interdependencies  to  accomplish  team  goals.”  Clearly,  this  definition 
assumes  that  active  team  goals  are  understood  and  recognized  by  all  team  members. 
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Given  this  assumption,  then  it  is  reasonable  to  differentiate  teamwork  types  based  on  the 
nature  and  extend  of  interdependencies. 

Regain  and  Elliott  develop  a  taxonomy  to  help  explicitly  differentiate  teamwork  types  in 
terms  of  interdependencies.  Their  taxonomy  is  derived  from  the  relationship  between  an 
individual  work  or  task  complexity  dimension  and  a  stage  model  of  human  work.  The 
stage  model  is  expressed  in  terms  of  four  cyclic  stages  [Information  (acquisition), 
Deliberation,  Resolution,  and  Action],  and  the  task  complexity  dimension  is  partitioned 
into  4  regions.  This  results  in  a  4  by  4  matrix  where  cell  descriptors  suggest  the  degree  of 
dependencies  that  may  be  involved  to  achieve  the  work  at  a  stage  according  to  its 
complexity.  The  most  complex  situation  consists  of  information  that  is  fuzzy 
(ambiguous,  imcertain);  deliberation  is  mental  model  based  and  involves  fuzzy,  complex, 
contingent  procedures;  resolution  requires  expert  judgment;  and  action  requires  expert 
level  performance.  At  the  simple  end  of  the  scale,  information  is  concrete  in  form; 
deliberation  involves  only  facts;  resolution  can  be  obtained  with  simple  rales;  and  action 
is  only  dependent  on  basic  abilities. 

The  final  element  of  the  Regian  and  Elliott  framework  is  a  six-factor  model  of  teamwork 
effectiveness.  The  factors  are:  allocate  tasks,  allocate  resources,  exchange  information, 
determine  strategy,  monitor  team  performance,  and  adaptive  problem  solving.  It  is  easy 
to  see  how  the  first  four  factors  relate  to  team  interdependencies.  Team  performance 
monitoring  is  clearly  a  meta-level  factor.  As  a  factor,  adaptive  problem  solving  maybe  a 
composite  of  the  first  four  factors.  However,  it  is  not  clear  how  Regian  and  Elliott  intend 
to  handle  the  dynamic  characteristic  of  this  factor. 

This  framework  has  been  established  to  support  theory-based  development  aimed  at 
formulating  training  intervention  strategies,  methods,  and  techniques  to  be  used  to 
facilitate  teamwork  for  different  types  of  teams  and  work  situations.  It  appears  to  be 
nicely  developed  and  well  suited  to  this  purpose.  Our  research  objectives  are  also 
concerned  with  the  effectiveness  of  team  performance,  but  the  focus  is  slightly  different. 
Just  as  training  methods  can  contribute  to  good  team  performance,  so  to  can  the  content, 
form,  and  behavior  of  the  interfaces  among  team  members  and  information  network 
technology,  including  embedded  tools.  Our  objective  is  to  understand  this  interfacing 
relationship  to  the  point  where  theory-based  design  principles  can  be  established  to  guide 
interface  development.  Ideally,  we  would  like  to  be  able  to  make  performance 
predictions  about  how  well  interfaces  developed  from  such  principles  impact  team 
effectiveness. 

Similar  to  Regian  and  Elliott,  we  approach  the  problem  from  a  skills  perspective.  Based 
on  our  assessment  of  the  emerging  digital  battlefield  and  concepts  of  operations,  we 
believe  that  the  user  interface  with  network  IT  must  be  able  to  support  expert-level 
teamwork.  Thus,  the  types  of  teams  we  are  most  interested  in  have  the  interdependencies 
suggested  by  the  high-end  column  in  the  Regian  and  Elliot  taxonomy.  Accordingly,  we 
hypothesize  that  the  effective  functional  purpose  for  the  joint  functional  interface  of  a 
network  IT  system  is:  to  serve  as  a  support  system  to  facilitate  expert-level  individual 
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and  team  problem  solving  performance.  Thus,  this  serves  as  a  definition  of  a 
collaborative  interface.  The  interface  is  a  support  system,  not  merely  an 
information/communication  channel,  (although  that  will  clearly  be  one  aspect  of  its 
support  function).  More  specifically,  it  is  a  support  system  for  expert  problem  solving 
behavior.  This,  of  course,  begs  the  question,  what  is  expert  individual  and  team  problem 
solving  behavior? 


E.  Characterizations  of  Expertise 

1.  Individual  expertise 

Expertise  has  been  characterized  as  know  how  (i.e.,  the  ability  to  establish  an  effective 
problem  characterization  and  adaptively  execute  it  with  real-time  modifications),  versus 
simply  knowing  that  (i.e.  knowing  facts,  rules,  and  procedures).  All  skilled  behavior  may 
be  regarded  as  dependent  on  basic  abilities,  training,  and  experience.  Experience,  often 
substantial  in  amount,  is  generally  regarded  as  essential  to  the  formation  of  expert-level 
skill.  In  this  sense,  it  is  a  defining  property  of  expertise.  It  applies  for  expertise  at  each 
stage  in  a  work  model  or  for  the  entire  constellation  of  activities  that  defines  work. 
Further,  it  suggests  that  to  understand  expert  behavior,  one  must  observe  experience 
workers  in  their  work  situation.  Retrospective  reports  and  other  approaches  are  not  likely 
to  be  satisfactory  because  much  expert  behavior  is  automatized  and  it  is  characterized 
below  the  awareness  level  of  the  performer. 

Over  the  past  several  years,  Klein  and  his  associates  have  been  stud5dng  expert  decision 
making  in  complex,  higji  dimensional  tasks,  such  as  fire  fighting,  neonatal  intensive  care, 
electric  power  control,  and  various  levels  of  military  command  activities  (e.g.,  Klein, 
1989;  Klein  et  al.,  1986;  Klein  et  al,  1993).  By  employing  various  observational  and 
cognitive  analysis  techniques,  these  researchers  have  succeeded  in  building  a  large  base 
of  empirical  data  on  decision  processes  “in  the  wild”  and  distilling  sound  inferenees  fi’om 
this  base.  Based  on  their  findings  they  have  advanced  a  recognition-primed  model  of 
decision  making  in  naturalistic  settings  to  characterize  expert  decision  making  behavior 
in  the  ill-structured  task  situations  upon  which  they’ve  focused.  Klein’s  overall 
characterization  of  the  perspective  emerging  firom  this  research  is  conventionally  labeled 
naturalistic  decision  making  (NDM). 

The  more  formal  or  abstract  aspect  of  this  work  is  Klein’s  Recognition-Primed  Decision 
(RPD)  model  of  decision  making,  which  we  shall  exploit  in  developing  our  workspace 
model  for  expert-level  task  behavior.  Within  the  framework  of  the  RPD  model,  Klein 
(1997)  develops  a  list  of  functional  behaviors  in  which  experts  engage  and  identifies 
methods  or  strategies  they  use  to  accomplish  them.  These  key  functional  activities;  (1) 
noticing  patterns,  (2)  seeking  information,  (3)  meaning  making,  and  (4)  managing 
imcertainties. 
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As  Klein  and  others  have  noted,  expert  performers  often  do  not  view  themselves  as 
making  decisions;  rather  they  simply  notice  patterns  that  are  meaningful  based  on 
experience.  This  is  the  essence  of  the  RPD  model.  When  the  recogmzed  pattern  needs 
elaboration  or  verification,  experts  seek  additional  information,  being  guided  by  the  initial 
recognition.  Part  of  the  verification  or  elaboration  process  may  involve  the  construction 
of  a  story  to  “see”  if  the  emerging  situational  understanding  makes  sense.  This  meaning 
making  activity  may  also  involve  prediction,  extrapolation,  and  the  use  of  meta-cogmtive 
reflection. 

Ambiguities  and  uncertainties  typically  abound  in  ill-structured  real-world  environments. 
The  final  activity  required  of  an  expert,  then,  is  to  manage  these  uncertainties.  Klein 
views  such  management  in  terms  of  (1)  knowing  when  to  accept  imcertainty  and  press 
ahead,  (2)  knowing  when  to  seek  more  information  to  eliminate  uncertainties,  and  (3) 
knowing  how  to  look  differently  at  the  situation  such  that  the  critical  uncertainties  simply 
vanish.  This  characterization  maybe  regarded,  at  least  loosely,  as  a  descriptive  work 
process  model  of  individual  expertise.  It  represents  a  description  of  the  general  properties 
of  expertise  that  are  made  manifest  in  each  specific  work  context.  The  details  of  each 
activity  will  be  different  for  each  job  episode.  Because  this  general  character  of  the  work 
process  is  evidenced  in  all  the  contexts  studied  by  Klein  and  his  colleagues,  it  is 
reasonable  to  consider  them  general  characteristics  of  expert  behavior.  Further  support 
for  this  position  comes  from  Dreyfus’s  account  of  skill  acquisition.  Dreyfus  and  Dreyfus 
(1986)  present  a  five-stage  model  of  skill  acquisition  that  includes  expert  behavior  as  the 
highest  level  of  skill.  They  identify  mtuiy  of  the  same  characteristics  of  expert  behavior 
as  theses  suggested  by  Klein.  Similar  observations  were  made  by  Chase  (1986)  and 
Chase  &  Simon  (1987)  in  their  studies  of  chess  masters. 
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2.  Team  Expertise 

It  is  generally  accepted  that  the  phenomenon  of  (individual)  expertise  exists  and  thus  is  a 
valid  subject  for  scientific  investigation.  This  is  true  even  though  many  may  also  believe 
the  construct  remains  to  be  adequately  parametrized.  The  concept  is  appealing  because  it 
seems  to  make  contact  with  a  quality  of  behavior  observed  in  real  life  that  is  not  captured 
by  a  more  analytic  or  rational  characterizations  of  decision  making  and  other  highly 
skilled  activities.  In  the  same  spirit,  it  may  be  useful  to  extend  the  notion  to  a  team  as  an 
entity.  What  do  we  mean  by  team  expertise? 

As  Regian  and  Elliott  point  out,  individuals  exhibit  behavior,  not  teams,  and  thus  in  some 
sense  a  team  cannot  be  said  to  have  expertise.  But  in  another  sense,  expertise  can  be 
viewed  as  a  team  property  based  on  coordination  and  synchronization  of  activities  (i.e. 
interdependencies).  Thordsen,  Klein,  and  Kyne  (1994)  have  viewed  teams  as  cogmtive 
entities  and  used  observational  strategies  to  investigate  team  decision  making.  Based  on 
this  work  they  have  advanced  a  model  of  team  decision  making  that  may  be  regarded  as  a 
description  of  properties  that  define  team  expertise.  Their  model  is  called  the  Advanced 
Team  Decision  Making  (ATDM)  model.  It  consists  of  four  constructs  that  summarize  a 
set  of  13  behaviors  believed  to  be  essential  to  high  performance  (expert)  teams.  The 
constructs  are:  (1)  Team  Competencies,  (2)  Team  Identity,  (3)  Te^  Cognition,  and  (4) 
Team  Metacognition.  Team  competencies  refer  to  both  individual  team  member  skills 
and  to  proficiency  in  common  coordinated  action  routines.  As  the  name  implies.  Team 
Identity  indexes  the  degree  to  which  individual  members  feel  connected  to  the  team. 

Team  Cognition  focuses  most  directly  on  joint  or  team  problem  solving,  decision  making, 
and/or  action  taking.  Behavior  markers  such  as  having  a  common  goal,  having  a 
common  picture  of  the  problem  situation,  recognizing  when  their  course  of  action  is  on- 
track,  off-track,  falling  behind,  etc,  identify  it.  Team  Metacognition  refers  to  behavior 
aimed  at  team  operations  themselves,  such  as  noticing  emerging  problems  and  taking 
responsibility  to  head  them  off,  correcting  problems,  and  helping  out  when  a  team 
member  seems  to  be  falling  behind. 

We  beheve  that  a  characterization  of  individual  and  team  performance  (problem  solving, 
decision  making,  action  taking)  in  terms  of  the  expertise  construct  provides  a  good  point 
of  departure  for  studying  the  interaction  between  team  interfaces  in  highly  connected 
network-based  work  environment  and  team  effectiveness.  A  more  traditional  approach 
would  view  team  behavior  firom  a  rational  choice  perspective,  and  assert  that  team 
members  would  systematically  search  for  relevant  information,  systematically  manage 
coordination  and  synchronization,  and  methodically  weigh  identified  alternatives  in  the 
process  of  identifying  and  taking  a  course  of  action.  Observations  of  high  achieving 
performers  in  demanding  real-world  situations  suggest  that  a  rational  choice  framework  is 
not  consistent  with  their  behaviors.  Rather,  the  expertise  view,  although  perhaps  less 
precise  and  not  as  thoroughly  developed,  better  reflects  actual  work  process  behavior. 
Given  our  stated  hypothesis  that  the  joint  functional  interface  for  an  IT-based  work 
environment  should  be  designed  as  a  system  to  support  adaptive  team  problem  solving. 
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. 

one  approach  to  establishing  design  requirements  for  the  interface  is  to  model  support 
aroxmd  what  is  needed  to  facilitate  expert  behavior. 


F.  Workspace  Models 

Expert  behavior  is  adaptive  and  partially  self-organizing.  Any  system  that  has  this 
characteristic  may  be  said  to  be  “open”  in  the  sense  that  multiple  degrees  of  freedom  for 
action  taking  remain  available  until  the  (human)  system  itself  decides  to  close  them.  This 
implies  that  whether  or  not  a  closed-loop  approach  is  employed  to  model  the  cogmtion  of 
the  human,  such  an  approach  (in  which  only  discrete  trajectories  can  be  modeled)  is 
insufficient  for  modeling  task  activities  themselves.  A  closed-loop  model  requires  that  all 
degrees  of  freedom  be  eliminated  except  for  those  that  support  specific  trajectories.  Since 
there  often  may  be  a  very  large,  possibly  infinite,  number  of  trajectories  available  to 
human  actors  in  a  given  context,  closed  loop  models  are  usually  developed  for  only  a 
small  set  of  characteristic  trajectories.  For  ill-structured,  complex  situations, 
observational  studies  indicate  that  garden  path  trajectories  are  rarely  followed.  Thus,  the 
utility  of  a  closed-loop  model  is  compromised  in  multi-factored,  ill-structured  situations. 
A  different  modeling  approach  is  needed  to  adequately  explore  expert  individual  and 
team  performance  in  these  situations. 

We  have  elected  to  follow  a  modeling  approach  which  focuses  attention  on  the 
collaborative  scenario  as  a  workspace.  This  approach  attempts  to  describe  the  boundaries 
that  specify  the  region  of  work  (i.e.,  the  “workspace”)  within  which  the  expert  develops  a 
course  of  action.  Rasmussen  ef  a/.  (1994)  and  Flach  (1996)  have  characterized  a 
workspace  in  terms  of  so-called  constraint  boundaries.  These  boundaries  may  be 
delineated  with  respect  to  a  munber  of  referential  or  indexical  parameters.  For  example, 
Rasmussen  has  used  high  level  abstract  constructs  such  as  workload  to  demarcate  the  type 
of  workspace  analyzed  in  his  cognitive  sj^tems  engineering  work  (Rasmussen  et  al, 
1994).  Flach  and  Warren  (1995)  employ  competing  goal  constraints  for  low-level  flight 
and  safety  to  generate  a  cohesive  depiction  of  a  workspace  for  safe  flight.  They  define  a 
workspace  in  terms  of  these  intentional  constraints  coupled  with  physical  constraints 
deriving  from  environmental,  structural,  and  avionic  factors.  To  date,  such  workspace 
description  has  proven  useful  for  specific  tasks  or  scenarios. 

However,  no  attempt  has  been  made  to  capture  a  general  view  of  work  processes  in  a 
single  workspace  model.  We  propose  to  express  a  theory  of  expert  performance  in  terms 
of  a  work  activity  model  represented  as  a  workspace.  Our  approach  takes  Klein’s  RPD 
model  of  individual  expertise  as  a  point  of  departure  and  develops  the  workspace 
description  in  a  different  manner. 

The  results  of  initial  model  development  are  presented  in  Figures  n.3  and  n.4,  which 
illustrate  our  approach  with  respect  to  individual  and  team  expert-level  performance, 
respectively.  The  two  models  have  essentially  identical  layouts  in  which  a  set  of  tree 
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structures  (the  left-to-ri^t  horizontal  extensions)  are  related  to  each  of  the  four  key 
functional  behaviors  id^tified  by  Klein  (and  listed  vertically).  Each  tree  originates  with 
one  of  Klein’s  key  behaviors  and  progresses  toward  an  executable  procedure. 
Intermediate  branches  in  the  tree  identify  such  things  as  object  of  focus  and  relative 
environmental  uncertainty  in  the  problem  space.  In  toto,  the  tree  set  captures  a  space  of 
possibilities  cast  from  the  perspective  of  expertise  patterns  discernible  in  the  best 
available  empirical  data. 

Experts  freely  navigate  through  this  tree  set.  No  course  is  prescribed.  However,  the 
relative  environment  state  variable  (i.e.,  level  of  imcertainty)  acts  as  a  bias  term  and  thus 
can  serve  as  a  pruning  mechanism  for  shaping  the  workspace.  Recall  that  managing 
uncertainties  was  one  of  the  key  factors  in  Klein’s  theory  of  individual  expertise.  We 
have  not,  however,  listed  uncertainty  management  as  one  of  the  four  primary  behavior 
classes.  It  is  explicitly  subsumed  in  the  model  within  some  of  the  intermediate  branches 
of  die  Action  Taking  tree.  This  is  a  deliberate  depictive  arrangement.  Because  we  are 
aiming  to  address  performance  (i.e.,  quality  of  action),  we  must  make  room  for  “action 
taking”  as  one  of  the  main  categories.  Uncertainty  management  is  not  represented  as  a 
root  for  a  primary  tree  structure  because  the  work  process  requisite  to  this  behavior  is 
expected  to  include  some  combination  of  the  four  root  work  process  activities  listed. 
Phrased  another  way,  we  have  treated  uncertainty  as  a  central,  quantifiable  control 
variable  in  our  model,  while  integrating  an  explicit  category  of  action  taking  which  can 
subsume  (e.g.)  the  end  state  of  a  simple  linear  trajectory  through  the  Klein  behaviors  (i.e., 
moving  from  top  to  bottom  in  the  figures)  and  /  or  those  behaviors  which  may  be 
recursively  “spun  off’  in  achieving  some  form  of  closure  to  the  activity  being  modeled. 
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Figure  II.3:  Workspace  Model  for  Individual  Expert  Behavior 

A  similar  tree  structure  is  used  to  represent  the  workspace  devoted  to  team  activities  by 
expert  team  members  (see  Figme  n.4).  This  workspace  is  identical  in  form  to  that  for 
individual  problem  work.  The  only  difference  is  in  the  focus  of  the  work  problem.  The 
assumption  is  that  basic  expert  work  processes  apply  at  the  team  level  as  well  as  at  the 
individual  level,  but  with  the  indicated  change  in  foci. 

In  some  sense  these  models  compile  the  work  process  theories  of  expertise.  They  leave 
open  the  trajectories  an  expert  may  take.  Thus,  they  do  not  provide  a  basis  for  predicting 
specific  performance  outcomes.  But  they  are  helpful  in  other  ways  and  they  suggest  other 
types  of  testable  hypotheses.  For  example,  the  model  suggests  that  experts  will  tend  to 
use  “best  practices”  when  they  do  not  have  more  insight  into  the  problem  situation.  This 
means  that  we  need  to  be  able  to  tease  apart  best  practices  from  other  bases  for  activities 
when  analyzing  process  data.  In  its  current  form,  the  workspace  model  predicts  that 
under  low  environment  state  uncertainty  experts  will  tend  to  commit  to  an  action  process 
that  contains  embedded  tests  to  insure  correct  xmderstanding  of  environment  state.  When 
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enviromnent  state  uncertainty  is  very  low  or  zero,  then  deliberate  testing  will  tend  to  be 
replaced  by  no  testing  or  only  incidental  testing  that  occurs  as  a  by-product  of  goal 
directed  activities.  Under  high  environment  state  uncertainty,  the  activity  pattern  will 
tend  to  change  to  a  more  cautious  probe-test-commit  sequence.  Process  data  can  be  used 
to  identify  these  patterns. 
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Figure  11.4:  Workspace  Model  for  Team  Expert  Behavior 

The  framework  provided  by  the  dieories  of  expertise  and  the  workspace  models  provide 
vital  guidance  for  what  activities  the  joint  functional  IT  interface  needs  to  support.  It 
does  not  suggest  specific  forms  of  interface  mechanization  since  they  will  change  based 
on  the  capabilities  and  limitations  of  the  specific  IT  platform(s)  implementing  a  given 
work  support  system. 
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G.  Summary 

1 

Section  H  has  presented  a  progressi  ve  explanation  of  the  background  to  CSTL’s 
conceptual  or  theoretical  “infrastructure.”  As  has  been  demonstrated,  the  key  construct  of 
“interface”  is  central  to  framing  our  research  goals.  We  have  presented  detailed  reasons 
for  questioning  the  problematically  individualistic  focus  of  interface  research  and 
development  to  date,  and  we  have  linked  this  to  the  prevailing  information  processing 
paradigm  and  the  mode  of  IT  deployment  upon  which  such  work  has  concentrated.  We 
have  recontextualized  interface  issues  in  terms  of  more  concrete,  more  quantifiable,  and 
hence  more  “scientific”  constructs  (performance,  expertise,  and  workspace).  These 
alternative  focal  constructs  have  been  introduced  in  a  progressive  fashion,  tracing  a 
logical  path  along  which  at  each  “waypoint”  we  have  applied  knowledge  of  the  current 
state  of  the  art  to  critically  analyze  our  central  topic  (interfaces).  Similarly,  at  each  such 
point  we  have  introduced  a  product  in  the  form  of  a  model  or  framework  which 
“leverages”  our  interest  in  interfaces  with  respect  to  the  given  issue  or  phenomenon,  hi 
Section  IH  we  shall  turn  our  attention  to  CSTL’s  IT  infrastructure  development  and  its 
application  to  demonstrate  interface  issues  and  problems  in  the  context  of  a  simulated 
distributed  BMC4I  exercise. 


III. 

PRO  JECT  DEVELOPMENT: 

PHYSICAL  INFRASTRUCTURE  ACTIVITIES 

This  project  was  initiated  in  March  1997.  During  this  reporting  period  we  have 
developed  a  framework  for  investigating  interactions  between  team  performance  and  the 
user  interface  with  the  IT  network  systems,  as  described  earlier.  The  complete  framework 
includes  a  conceptual  description  of  a  user  interface  in  a  highly  connected  IT-based  work 
environment,  along  with  work  process  theories  of  team  and  individual  expertise  and 
associated  workspace  models.  In  addition,  we  have  also  completed  the  first  phase  in  the 
development  of  an  IT  network  infrastructure  that  can  be  used  to  support  empirical  testing 
of  team  performance  in  an  IT-based  work  place.  A  detailed  description  of  this  work  is 
presented  in  this  section. 

The  Collaborative  Systems  Technology  Laboratory  (CSTL),  Fitts  Human  Engineering 
Division  (AL/CFH),  Armstrong  Laboratory,  Wright-Patterson  AFB  OH,  is  the  newest 
laboratory  unit  widiin  the  division’s  Collaborative  Systems  Technology  branch  (CFHI). 
One  of  this  laboratory’s  missions  is  to  explore  those  human  performance  issues 
surrounding  the  utilization  of  advanced  information  technology  (IT)  in  support  of 
distributed  joint  operations.  This  document  describes  both  the  context  for  and  eonduct  of 
an  active  demonstration  (hereafter  ealled  the  "BMC4I  demo")  developed  to: 

•  Instantiate  and  test  initial  CSTL  conceptual  models  relevant  to  analysis  of 
collaborative  systems 

•  Test  simulation  applications  of  the  initial  CSTL  technological  infiustructure  (e.g., 
networked  eomputers  and  collaborative  software) 

•  Demonstrate  CSTL’s  ability  to  simulate  key  elements  of  operational  work  domains 

•  Spotlight  problematical  issues  surrormding  the  application  of  networked  /  distributed 
information  technology  in  support  of  joint  operations’  BMC4I 

•  Evaluate  any  constraints  or  limitations  in  readily-available  COTS  software  designed 
to  support  multi-user  collaboration  (i.e.,  “groupware”) 

At  the  time  of  the  initial  BMC4I  demo  exhibition,  only  approximately  50%  of  CSTL's 
initial  equipment  inventory  had  been  received  and  set  up.  As  a  result,  the  BMC4I  demo 
had  to  be  outlined,  plarmed,  and  executed  using  only  those  hardware  and  software 
resources  already  in  place.  The  effects  of  this  situation  are  discussed  more  fully  in  the 
Technical  Background  and  Limitations  and  Constraints  sections  below,  hr  spite  of  the 
infiustructure  restrictions,  CSTL  personnel  succeeded  in  setting  up  and  presenting  a 
demonstration  which  illustrated  problematieal  issues  in  joint  force  BMC4I  operations  and 
some  features  of  tbe  COTS  technologies  presently  available  to  meet  these  challenges. 


A.  Programmatic  Background 
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1.  Background 

a.  Networked  BMC4I  as  a  “System  of  Systems" 

CSTL’s  agenda  is  to  explore  the  means  by  which  warfighter  engagement  with  complex 
information  systems  can  be  fi’amed,  analyzed,  and  constructively  modified.  Battlespace 
management  C4I  (BMC4I)  concepts  continue  to  evolve  in  response  to  post-Cold-War 
exigencies,  technological  innovations,  and  the  U.S.  military’s  increasing  emphasis  on 
joint  force  operations.  One  key  focus  in  this  evolutionary  development  is  information 
technology  (TT)  and  its  role  in  future  military  missions.  A  force’s  IT  infi'astructure  is 
now  seen  as  an  operational  componeiit  subject  to  offensive  and  defensive  manipulations, 
and  “cyberspace”  is  now  seen  as  a  virtual  landscape  worthy  of  tactical  and  strategic 
concern.  Broadly  stated,  these  two  perspectives  comprise  the  still-coalescing  area  of 
information  warfare  (IW). 

The  application  of  advanced  IT  in  support  of  joint  operations  entails  a  technical 
infi^tracture  capable  of  providing  persoruiel  the  tools  needed  to  function  as  a 
coordinated  whole  even  though  they  may  be  located  throughout  a  theater  of  operations  or 
all  around  the  planet.  The  most  basic  such  tools  must  afford  participants  mutual  access  to 
each  other  (i.e.,  communications)  and  uniform  access  to  information  and  knowledge 
bases  (i.e.,  information  retrieval).  Information  technologies  supporting  these  fundamental 
functions  have  been  developed  and  deployed  for  decades.  More  recent  innovations  have 
now  laid  the  foimdation  for  further  leveraging  IT  by  allowing  for  dynamic  interactions 
with  data  streams  and  information  elements  (as  opposed  to  the  relatively  passive  nature  of 
information  retrieval).  In  the  civilian  realm,  this  development  parallels  the  evolution 
toward  increasingly  interactive  Internet  usage,  as  illustrated  by  the  progression  fi'om  FTP 
and  Usenet  toward  the  World  Wide  Web  (WWW)  and  online  multimedia. 

One  illustrative  eonstruct  critical  to  emerging  DOD  coneepts  on  employing  networked  IT 
is  the  system  of  systems  (SOS).  This  construct  is  strongly  associated  with  Admiral 
William  Owens,  who  popularized  it  as  a  means  for  describing  the  complex  interplay  of 
distinguishable  systemic  elements  to  realize  an  overall  operational  system.  To  given  an 
example  with  respect  to  achieving  battlespaee  information  dominance,  Owens  (1995a; 
1995b;  1995c),  outlined  three  ensembles  of  systems  contributing  to  an  SOS  architecture 
for  theater  operations  --  battlespace  awareness,  advanced  C^I,  and  precision  force  use. 
The  focus  of  the  work  reported  herein  is  the  central  (C^I)  component  of  Owens’ 
breakdown  --  i.e.,  the  system  "...that  eonverts  the  information  derived  fi'om  battlespace 
awareness  into  deeper  knowledge  and  understanding  of  the  battle  space..."  and  in  turn 
"...converts  the  understanding  of  the  battlespace  into  missions  and  assignments  designed 
to  alter,  control,  and  dominate  that  space."  (Owens,  1995a,  p.  38)  As  will  be  seen  later, 
the  BMC4I  demo  was  crafted  to  concretely  illustrate  technical  aspects  of  these  processes 
of  conversion. 


b.  CSTL ’s  Focus:  Interfaces  within  the  BMC4I  System  of  Systems 
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•  • 


Given  its  human  factors  /  cognitive  engineering  mission,  CSTL  must  match  its 
programmatic  focus  to  those  system  of  systems  issues  which  involve  the  engagement  of 
humans  and  technologies.  Current  DARPA  IT  initiatives  address  networked  operations 
in  terms  of  SOS  architecture,  as  illustrated  in  Figure  ni.l .  Nodes  (N)  represent  computers 
/  workstations.  They  are  linked  to  each  other  via  data  networks  whose  “hubs”  are  routers 
(R).  A  given  node  may  be  considered  an  element  of  multiple  distinguishable  systems 
(composite  networks),  depending  on  which  router(s)  and  other  node(s)  are  considered 
participants.  Two  such  overlapping  systems  (1  and  2)  are  delimited  by  the  dotted  lines  in 
Figure  in.1. 


System  of  Systems: 
No<tes,  Links,  Sy^ems 


R  s  Router 


/^jplication 

Coinponenls 


Middleware 


Oy  Kernel 


Figure  ni.l:  DARPA’s  SOS  Layout  for  Networked  IT 
Source:  Shrobe  (1996) 


However,  the  illustrated  DARPA  application  of  the  SOS  concept  addresses  Only  the 
technological  infrastructure.  Because  CSTL  research  must  accoimt  for  the  human  aspect 
of  SOS  operations,  we  must  extend  such  an  SOS  analysis  to  more  explicitly  show  the 
human-technology  relationships.  Online  interactivity  emphasizes  and  taxes  the  means  by 
which  users  or  operators  engage  complex  information  systems  —  i.e.,  the  interfaces 
through  which  they  must  work.  The  efficiency  and  effectiveness  gains  firom  interface 
innovations  on  individual  workstations  (e.g.,  windowing  environments)  are  now  being 
sought  with  respect  to  networked  IT  applications.  However,  such  payoffs  require  a 
cogent  research  and  development  effort,  and  such  cogency  must  entail  a  coherent  notion 
of  those  “interfaces”  which  will  serve  as  the  focus  of  CSTL  projects. 


When  addressing  joint  operations  via  IT  networks,  it  quickly  becomes  apparent  that  the 
conventional  notion  of  “interface”  (the  set  of  mechanisms  mediating  interactivity  of  an 
individual  operator  and  a  particular  device  or  workstation)  is  insufficient.  For  one  thing, 
network  operations  still  require  that  actors  engage  their  assigned  workstations  via  such 
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conventional  interface  facilities.  Phrased  another  way,  “net-workers”  use  their  individual 
interfaces  to  control  their  “windows”  into  a  broader  system  so  as  to  interact  with  others. 
The  notion  of  the  individual’s  interface  is  therefore  still  operant.  The  real  concern  is  what 
additional  concept(s)  of  “interface”  must  be  introduced  to  span  the  range  of  issues 
surrounding  human  performance  in  distributed  or  networked  missions. 

Such  conceptual  iimovation  is  difiBcult  owing  to  the  necessity  of  sorting  out  the  complex 
interdependencies  among  individual  and  collective  units  of  analysis  (e.g.,  single  users  vs. 
teams;  single  computers  vs.  entire  networks).  For  one  thing,  whole  /  part  decomposition 
cannot  neatly  segregate  the  subject  matter  into  discrete  hierarchical  layers  or  levels. 

Team  performance  is  still  contingent  upon  relative  individual  perfomance,  and  overall 
networked  system  performance  is  contingent  upon  specific  subsystem  performance 
(although  in  both  cases  the  former  is  not  strictly  reducible  to  the  latter). 

Furthermore,  any  such  “hierarchical”  approach  is  confounded  by  the  fact  that  networked 
task  activities  do  not  entail  unique  mappings  of  individual  acts  to  one  or  another  subset  of 
the  technological  infrastructure.  Any  joint  or  mutual  task  accomplished  via  the  overall 
network  /  system  is  pursued  by  individuals  engaging  their  respective  workstations.  To 
give  an  example,  people  can  engage  in  an  online  “chat  room”  discussion  via  their  Web 
browser  and  desktop  PC.  It  is  reasonable  to  study  the  overall  interaction  among  multiple 
such  participants,  but  the  nature  and  course  of  their  interaction  cannot  be  isolated  from 
the  fact  that  each  one  of  them  is  wrestling  with  the  particular  affordances  of  the  computer 
sitting  before  them.  Conversely,  it  is  reasonable  to  study  the  specific  interactions 
between  individual  users  and  their  PC’s,  but  these  actions  cannot  be  isolated  from  the  fact 
that  they  are  collectively  engaging  in  a  shared  conversational  space. 


2.  The  CSTL  Conceptual  Model  for  Addressing  Collaborative  Systems 

In  an  attempt  to  account  for  these  and  other  factors,  CSTL  has  developed  a  preliminary 

conceptual  model  of  human-system  interfaces  relevant  to  networked  IT  systems.  The 

rationale  for  devising  such  a  model  is  that: 

1 .  The  scope  of  critical  information  technology  (IT)  applications  is  evolving  from  an 
earlier  focus  on  single-user  /  standalone  implementations  toward  multi-user  / 
networked  implementations. 

2.  Given  our  division’s  hiunan  factors  /  cognitive  engineering  mission,  pur  focal  concern 
would  be  the  interface(s)  via  which  humans  interact  with  their  respective  workstations 
and  (via  their  respective  workstations)  with  collaborators  in  the  context  of  a  joint 
operation. 

3.  Models  and  oflier  analytical  devices  addressing  the  single-user  /  standalone 
implementation  modality  do  not  leverage  issues  delineated  with  respect  to  interactions 
among  multiple  actors  and/or  multiple  workstations. 

4.  Models  and  other  analytical  devices  addressing  IT-supported  collaboration  do  not 
leverage  issues  pertaining  to  the  individual’s  interaction  with  his  /  her  workstation. 


24 


As  a  result,  the  pursuit  of  CSTL’s  mission  required  that  we  first  attempt  to  identify  and 
delimit  the  relationships  between  individual  and  collective  “foci”  on  information 
technology  --  particularly  as  they  pertain  to  human-computer  interfaces.  Conventionally, 
the  term  “interface”  has  been  applied  to  a  scenario  of  one  user  to  one  artifact,  as  in  the 
case  of  desktop  computers.  Upon  closer  examination,  one  realizes  that  the  user  interacts 
with  multiple  distinguishable  subunits  of  the  composite  hardware  /  software  suite  which  a 
typical  PC  represents.  We  speak  of  the  “interface”  to  a  specific  application  software 
package  (e.g.,  Microsoft  Word),  the  “interface”  to  the  operating  system  (e.g.,  the 
Macintosh),  and  the  “interface”  to  the  hardware  itself  (e.g.,  keyboard  and  mouse).  Insofar 
as  the  user  engages  these  distinct  elements  as  one  suite  of  mechanisms,  it  does  little  harm 
to  colloquially  consider  this  suite  as  a  unary  whole  (i.e.,  “the  interface”).  For  the 
purposes  of  scientific  analysis  and  experimentation,  it  is  necessary  to  impose  a  stricter 
accoimting.  Such  an  accounting  underlies  the  CSTL  conceptual  model  for  an  individual 
interface  illustrated  in  Figure  in.2. 


Figure  in.2:  CSTL  Conceptual  Model  for  the  Individual  Workstation  Interface 

Figure  III.2  covers  the  differentiable  aspects  of  the  “interface”  between  an  individual  user 
and  his  /  her  computer  workstation.  The  3-layer  description  of  the  workstation  is  a 
simplified  variant  of  the  5-layer  DARPA  node  architectme  outlined  in  Figure  m.  1 .  To 
address  multiple  such  user  /  computer  dyads  participating  in  a  larger  context  (as  in 
networked  BMC4I),  it  becomes  necessary  to  specify  how  the  elements  of  the  individual 
case  relate  to  the  collective  case.  The  preliminary  CSTL  model  accomplishing  this 
expanded  scope  is  illustrated  in  Figure  in.3. 
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Figure  in.  3  depicts  a  progression  from  specific  hardware  (the  lowest  “layer”)  to  the 
software  appUcations  supporting  collaborative  tasks  in  networked  environments 
(“groupware”).  The  vertical  ordering  of  these  layers  is  not  intended  to  connote  some 
strict  hierarchical  segregation.  Instead,  it  lays  out  the  relative  dependency  of  elements 
required  to  build  up  and  exploit  networked  information  technologies.  The  lowermost  3 
layers  comprise  the  collection  of  elements  required  to  realize  a  single-user  workstation 
(hardware,  operating  system,  and  application  software).  Individual  users  engage  this 
subset  of  the  network  topology  in  any  case,  and  the  “interface”  elements  shown  represent 
the  conventional  usage  of  the  term.  The  uppermost  3  layers  represent  the  collection  of 
elements  required  to  “enhance”  an  individual  workstation  to  interoperate  with  others  via  a 
network.  It  is  this  additional  set  of  elements  which  users  utilize  in  collaborative 
interactivity. 


Figure  IIL3:  The  CSTL  Model  of  Networked  Collaborative  Systems 


With  respect  to  the  CSTL  agenda,  the  model  in  Figure  in.3  is  an  orderly  layout  of  the 
various  aspects  of  “interface”  as  they  pertain  to  networked  collaborative  systems.  It 
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should  be  noted  that  Figure  in.3  differs  from  the  very  similar  Figure  n.2  in  making  a 
distinction  between  two  aspects  of  the  joint  fimctional  interface.  Participants  in 
collaborative  operations  effectively  engage  the  technology  and  each  other  through  three 
distinguishable  “interfaces”.  The  first  (denoted  “interface”  in  Figure  in.3)  is  the 
conventional  interface  between  operator  and  workstation.  The  second  (“joint 
instrumental  interface”)  denotes  those  mechanisms  through  which  collaborators  engage 
the  objects  of  their  joint  effort.  The  third  (“joint  communication  /  coordination 
interface”)  denotes  those  mechanisms  through  which  collaborators  engage  each  other  to 
(e.g.)  exchange  and  clarify  messages. 

The  reason  for  distinguishing  among  these  three  types  of  “interface”  is  that  analysis  of 
joint  networked  operations  requires  some  means  for  disentangling  the  confusing  overlaps 
and  interdependencies  among  each  operator’s  access  to  and  interaction  with  their 
immediate  toolset  (workstation),  their  joint/  virtual  product  (shared  information 
artifacts),  and  each  other.  The  preliminary  CSTL  model  accomplishes  this  end,  and  it 
will  serve  as  the  main  explanatory  device  for  introducing  and  discussing  the  BMC4I 
demonstration. 
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3.  The  CSTL  Model  versus  the  DARPA  System  of  Systems  Model 

As  was  noted  for  Figure  in.2,  the  layout  of  Figure  in.3  is  a  variant  of  (rather  than  an 
exclusive  alternative  to)  the  DARPA  system  of  systems  layout  in  Figure  HI.  1 .  The 
Network  Ihfrastmcture  of  Layer  4  subsumes  some  elements  of  the  router  and  “net 
interface”  elements  of  the  DARPA  breakdown.  The  Network  Protocols  of  Layer  5  do  not 
have  an  explicit  analogue  in  the  DARPA  model  to  the  extent  they  are  operant  for  the 
routers.  One  could  make  a  case  that  to  the  extent  they  are  operant  on  individual 
workstations  (nodes),  they  are  subsumed  within  the  “net  interface”  element  within  the 
DARPA  node  architecture. 

The  Groupware  that  comprises  Layer  6  would  be  subsumed  under  the  applications  portion 
of  the  DARPA  node  architecture,  because  it  connotes  software  which  ultimately  runs  on 
an  individual’s  workstation.  Our  rationale  for  distinguishing  a  separate  Groupware 
component  does  not  contradict  the  DARPA  model.  For  the  purely  technological 
orientation  evidenced  in  Figure  HI.  1,  the  subsumption  of  group-oriented  applications 
within  the  general  class  of  applications  is  reasonable  and  accurate.  In  shifting  to  the 
human-oriented  stance  necessary  for  CSTL’s  mission,  it  is  useful  to  make  a  logical 
discrimination  between  software  supporting  individual  tasks  and  software  supporting 
te^  operations,  and  this  explains  the  distinction  between  our  Layers  3  and  6. 

This  last  point  illuminates  the  key  difference  between  the  DARPA  SOS  model  and  the 
CSTL  model.  The  former  is  framed  with  respect  to  the  technology  per  se,  and  the  latter  is 
framed  with  respect  to  operational  interdependencies  vis  a  vis  the  human(s).  Recognizing 
this  contrast  of  perspectives  (as  opposed  to  a  conflict  in  meaning)  allows  us  to  complete 
the  comparison  of  the  two  models.  The  DARPA  SOS  model  usefully  illustrates  that 
multiple  permutations  of  nodes  (linked  via  routers)  comprise  multiple  distinct 
technological  systems.  To  the  extent  that  these  distinguished  systems  are  comprised  of 
elements  which  are  themselves  interconnected  (directly  or  indirectly),  all  nodes  can  be 
descriptively  subsumed  within  some  total  “netspace”,  just  as  a  number  of  distinguishable 
networks  are  descriptively  lumped  together  as  “the  Internet.”  Such  a  totalizing 
subsumption  can  negate  the  utility  of  a  purely  technological  perspective,  because  this 
“netspace”  becomes  in  effect  a  featureless  backdrop  to  the  phenomenon  of  interest. 
Phrased  another  way,  such  a  totalizing  subsumption  robs  the  construct  “netspace”  of  any 
capacity  for  analytical  “leverage”  with  respect  to  CSTL’s  purposes. 

This  explains  why  (for  our  purposes)  we  have  constructed  the  CSTL  model  from  a  more 
functional  or  operational  perspective.  The  interface  between  a  specific  user  and  a 
particular  workstation  is  a  precisely  delineable  combination  of  technologies.  However, 
the  technologies  employed  to  realize  what  we  have  termed  the  Joint  Instrumental 
Interface  and  the  Joint  Communication  /  Coordination  Interface  need  not  be  identical 
“systems”  as  defined  in  Figure  IH.  1 .  Collaborators  might  simultaneously  share 
information  or  data  (Joint  Instrumental  Interface)  via  one  such  “system”  while 
communicating  person-to-person  (Joint  Conummication  /  Coordination  Interface)  via 
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another.  Furthermore,  these  distinctions  might  be  mirrored  by  distinctions  between 
technological  infrastructures,  as  when  one  shares  data  via  computer  network  and 
conversation  via  telephone  in  getting  remote  technical  support  (e.g.,  from  a  software 
vendor).  To  usefully  analyze  human  factors  aspects  of  distributed  operations  CSTL  must 
be  able  to  discriminate  among  the  ‘Virtual”  or  “logical”  networks  collaborators  utilize, 
whether  or  not  these  distinguishable  interactional  channels  can  be  mapped  directly  or 
uniquely  onto  the  sets  of  node  intercoimections  emphasized  in  Figure  in.l. 

As  such,  we  would  claim  that  there  are  no  substantive  clashes  between  the  DARPA  SOS 
model  and  the  CSTL  model  of  Figure  1II.3.  Both  play  on  the  concept  of  “system  of 
systems.”  The  DARPA  model  lays  out  its  SOS  architecture  with  exclusive  regard  to 
technical  infrastructure,  and  the  CSTL  model  does  so  with  regard  to  operational 
interdependencies  vis  a  vis  the  human  user  /  operator.  Phrased  another  way,  moving 
“upward”  through  the  DARPA  model  entails  a  progressive  subsumption  of  technological 
subsystems,  while  a  similar  traversal  of  the  CSTL  model  entails  a  progressive 
subsumption  of  joint  human-machine  subsystems. 

fri  the  following  section,  the  BMC4I  demo  apparatus  will  be  introduced  and 
contextualized  as  an  instantiation  of  the  CSTL  conceptual  model  of  Figure  in.3.  Later,  in 
Figure  HI.  10,  we  will  provide  a  complete  accoimting  of  the  specific  BMC4I  demo 
apparatus  with  respect  to  this  model. 


B.  Apparatus  for  the  BMC4I  Demo 

This  section  will  outline  the  hardware  and  software  elements  providing  the  foundation  for 
the  CSTL  BMC4I  demonstration.  These  elements  will  be  related  to  the  six  layers  of  the 
preliminary  CSTL  model  illustrated  in  Figure  in.3. 


1.  Hardware 

a.  Workstations  (Layer  1  in  Figure  III.3) 

The  initial  CSTL  laboratory  infrastructure  will  feature  six  Intergraph  TDZ-series  graphics 
workstations  (each  with  two  large  monitors)  and  five  Silicon  Graphics  workstations. 
These  stations  will  enable  CSTL  to  emulate  the  heterogeneity  of  the  wide-area  networks 
employed  in  joint  operations.  Large  group  display  of  information  from  one  or  more  of 
these  stations  will  be  done  with  two  InFocus  Digital  Light  Processing  (DLP)  projectors 
and  one  or  more  large  projection  screens.  Each  of  the  Intergraph  stations  will  be 
equipped  with  Connectix  QuickCam  digital  cameras  to  enable  inter-station  video 
conferencing. 

b.  Network  Infrastructure  (Layer  4  in  Figure  III.  3) 
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These  computers  will  be  linked  via  a  100Base-T  CSTL  internal  LAN.  A  lO/lOOBase-T 
switch  will  link  the  CSTL  internal  LAN  to  the  AL/CFH  LAN  and  external  (e.g.,  Internet) 
networks 

As  illustrated  in  Figure  in.4,  the  two  Intergraph  stations  (and  the  DLP  projector)  were 
linked  via  the  100Base-T  CSTL  internal  LAN.  The  video  data  streams  from  the 
QuickCams  as  well  as  all  other  data  commimications  between  these  stations  were  handled 
over  the  internal  LAN.  A  third  Intergraph  TDZ-410  served  as  die  large-scale  display 
driver,  sharing  one  common  data  display  with  the  two  other  Intergraph  stations  and 
pumping  this  image  to  die  DLP  projector. 

Of  the  two  primary  software  packages  employed  (cf.  &e  Software  section  below),  one 
(NetMeeting)  operated  exclusively  between  the  Intergraph  stations  over  the  100Base-T 
internal  LAN.  The  other  (PowerPoint)  had  to  be  accessed  independently  by  each  station 
from  the  (lOBase-T)  AL/CFH  LAN.  This  extended  access  link  to  PowerPoint  had  to  be 
mediated  by  the  lO/lOOBase-T  switch  serving  as  the  gateway  between  the  CSTL  and 
AL/CFH  LANs. 

c.  Hardware  Summary 

At  the  time  of  the  first  BMC4I  exhibition,  CSTL  was  capable  of  employing  foxir 
Intergraph  workstations  (with  QuickCams),  the  internal  LAN,  the  external  LAN 
connections,  and  one  of  the  DLP  projectors.  These  elements  comprised  the  infrastructure 
basis  for  the  BMC4I  demo.  Three  of  the  Intergraph  stations  and  all  the  other  available 
elements  cited  were  utilized  in  the  BMC4I  demo.  For  the  purposes  of  the  BMC4I  demo, 
these  elements  were  configured  as  illustrated  in  Figure  III.4. 
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Figure  111.4:  CSTL  Hardware  Infrastructure  for  the  BMC4I  Demo 


2.  Software 

a.  Operating  Systems  (Layer  2  in  Figure  III.  3) 

All  3  Intergraph  workstations  were  running  Microsoft  Windows  NT  4.  All  software 
utilized  in  the  BMC4I  demonstration  ran  atop  the  Windows  NT  operating  system.  In  the 
future,  CSTL’s  heterogeneous  internal  LAN  will  provide  the  eapacity  for  demonstrations, 
experiments,  and  studies  more  closely  modeling  the  problematical  diversity  of  wide-area 
networks. 

b.  Individual  Application  Software  (Layer  3  in  Figure  III.  3) 

Most  of  the  information  displays  presented  at  each  station's  monitor(s)  were  mocked  up 
using  Microsoft  PowerPoint  4.  Each  individual  “window”  was  actually  a  discrete 
PowerPoint  file  comprised  of  one  or  more  slides  containing  the  appropriate  data.  To  set 
up  the  ^pearance  of  a  coherent  display,  all  the  PowerPoint  files  were  opened  and  their 
respective  windows  tiled  on-screen  to  achieve  the  desired  arrangement. 

c.  Networking  Protocols  (Layer  5  in  Figure  III.  3) 


The  networking  among  the  Intergraph  workstations  was  accomplished  using  the  TCP  /  ff 
protocol  over  the  CSTL  100Base-T  internal  LAN.  The  networking  required  to  access 
Microsoft  PowerPoint  finm  the  AFRL/CFH  LAN  was  accomplished  atop  the  Novell 
NetWare  IPX  which  serves  as  that  network’s  resident  “logic.” 

.  d.  Groupware  (Layer  6  in  Figure  III.  3) 

The  shared  collaborative  whiteboard  and  desktop  video  conferencing  functions  were 
accomplished  using  Microsoft  NetMeeting.  This  software,  available  free  of  charge  from 
Microsoft,  allows  a  combination  of  shared  whiteboard,  application  sharing  and  two-way 
video  conferencing  on  a  point-to-point  basis.  The  third  Intergraph  station  (the  one 
driving  the  projector)  also  employed  NetMeeting's  application  sharing  feature  as  a  means 
for  obtaining  updated  information  fields  (cf.  the  Information  section  below)  to  be 
presented  on  the  large-scale  projection  surface. 

The  only  windows  presented  at  the  stations’  monitors  which  were  not  mocked  up  in 
PowerPoint  were  the  two  video  windows  and  the  shared  whiteboard  (which  were 
presented  via  Microsoft  NetMeeting).  These  windows  were  presented  exactly  as 
NetMeeting  provided  them  (with  some  on-screen  tiling  and  alignment  to  blend  them  in 
with  the  majority  PowerPoint  displays).  A  more  detailed  review  of  the  BMG4I  demo’s 
information  displays  will  be  given  below  in  Section  V  (Information  Displayed,  Shared, 
and  Manipulated). 


C.  The  BMC4I  Demonstration  Task  Scenario 

1.  Overview 

The  BMC4I  demo  was  constructed  on  the  basis  of  a  mock  battlespace  scenario.  The 
scenario  was  generated  for  the  purposes  of  the  demonstration,  and  it  does  not  reflect  any 
specific  past  or  planned  circumstances.  As  illustrated  in  Figure  in.6  (in  section  V.),  blue 
forces  deployed  along  the  southern  and  eastern  extremities  of  the  Arabian  Peninsula,  as 
well  as  northward  into  southwestern  Asia  and  southward  in  Somalia,  were  confronted  by 
a  red  force  westward.  The  red  force  was  comprised  of  two  primary  groupings.  The  area 
of  engagement  with  the  northern  grouping  (centered  in  Syria)  was  designated  “Alpha 
sector”.  The  area  of  engagement  with  the  southern  grouping  (centered  in  Sudan)  was 
designated  “Bravo  sector”. 

The  two  workstations  used  in  the  demo  supported  the  roles  of  one  USAF  operator  (“Eagle 
Vantage  Center”)  and  one  Army  operator  (“Fort  Apache”).  Each  of  the  operators  was 
assigned  to  monitor  incoming  event  data  from  his/her  respective  portion  of  the 
battlespace  (e.g..  Army  =  ground;  USAF  =  air).  Based  on  this  incoming  data,  and  in 
coordination  with  the  other  player,  each  operator  was  tasked  to  compile  summary 
information  on  the  total  (i.e.,  ground  +  air)  situation  in  one  of  the  sectors  (Army  =  Alpha; 
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USAF  =  Bravo).  This  compiled  information  was  to  be  posted  to  a  sector  situation 
summary  which  was  then  forwarded  to  a  mock  command  center. 

The  three  components  of  the  BMC4I  demo  (USAF,  Army,  and  CINC)  were  represented 
by  three  stations  set  up  in  the  CSTL  facility.  The  two  operators  were  seated  at  hitergraph 
workstations  side-by-side.  This  close  proximity  was  judged  reasonable  for  providing 
visitors  a  concise  introduction  and  overview  of  the  demonstration.  An  acoustics- 
dampening  partition  was  inserted  between  the  two  stations,  to  reinforce  the  operators’ 
need  to  communicate  through  the  audio  /  video  conferencing  capabilities  of  the  demo’s 
groupware  component  (NetMeeting).  The  mock  CINC  station  Consisted  of  a  large-screen 
projection  of  the  Summary  Situation  Display  and  a  conference  table,  located  in  another 
section  of  the  CSTL  facility.  This  arrangement  is  illustrated  in  Figure  in.5. 


Figure  111.5:  Layout  of  the  BMC4I  Demo  Stations 


2.  Issues  to  be  illustrated  by  the  BMC4I  Demonstration 

The  demo’s  scenario  layout  was  contrived  to  illustrate  several  issues  which  in  one  form 
or  another  had  proven  problematical  in  prior  joint  force  operations.  The  following 
sections  will  briefly  discuss  these  issues. 

a.  Distributed  operations  require  attention  to  inter-operator  coordination 

The  operators  in  the  BMC4I  demo  were  required  to:  (1)  attend  to  incoming  data  for  tiieir 
designated  battlespace  focus  (air  vs.  ground);  (2)  compile  data  from  their  respective 
incoming  data  streams;  (3)  exchange  data  with  the  other  operator;  (4)  assemble  a 
composite  data  product  for  their  designated  battlespace  area  (Alpha  vs.  Bravo  sector);  and 
(5)  accrete  this  composite  data  product  to  the  shared  information  space.  Of  these  five 
basic  fimctions,  only  the  first  two  could  be  comprehensively  accomplished  by  one 
operator  in  isolation.  Functions  (3)  and  (4)  required  transfer,  comparison,  and/or 
coordination  of  data  items  with  the  other  operator.  Function  (5)  required  that  ttie  two 
operators  take  turns  in  accreting  their  respective  data  products  to  the  shared  information 
space.  The  coordination  and  tum-taking  overhead  required  for  functions  (3)-(5)  are 
typical  for  distributed  or  “groupware”  software  packages.  Because  this  overhead  is 
reflected  by  an  additional  cognitive  /  perceptual  “load”  on  each  individual  operator,  there 
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is  an  issue  as  to  the  extent  to  which  positive  collective  performance  effects  are  mitigated 
by  negative  effects  of  heightened  individual  workload. 

b.  "  Vertical  stovepipes  ’’for  data  streams  restrict  lateral  data  sharing 

The  term  “stovepipe”  has  been  used  for  years  to  connote  a  data  or  information  stream 
which  is  coherent  and  relatively  impervious  to  inputs  or  outputs  orthogonal  to  the  primary 
direction(s)  of  flow.  A  “vertical”  stovepipe  is  one  where  die  primary  directions  of  flow 
are  “up”  and  “down”  ttirough  a  hierarchical  organization  (e.g.,  as  illustrated  by  the  classic 
chain  of  command).  In  the  case  of  the  BMC4I  demonstration  setup,  there  were  two 
vertical  stovepipes  for  data  --  the  USAF  and  Army  data  streams  for  air  and  groimd  events, 
respectively.  Our  basic  demo  layout  afforded  both  operators  access  to  both  of  these  data 
streams,  but  this  is  not  representative  of  the  situation  for  many  joint  force  operations.  In 
all  too  many  cases,  operators  from  different  services  and/or  working  with  distinct  data 
streams  have  direct  access  to  only  their  own  (service’s,  mission’s)  data  flow. 

Where  such  “stovepiping”  applies,  lateral  sharing  of  data  /  information  (e.g.,  between 
USAF  and  Army;  between  air  and  ground  components)  becomes  a  major  problem. 

Owing  to  historical  differences  in  (e.g.)  technical  implementations,  deployment  styles, 
nomenclature,  data  formats,  resource  allocations,  and  procedures,  joint  force  components 
up  to  and  including  entire  services  have  foimd  it  anything  but  straightforward  to 
communicate  and  collaborate  among  themselves. 


c.  "Stovepipes  ’’  can  occur  with  respect  to  elements  /factors  other  than  data  streams 

The  BMC4I  demonstration  illustrated  a  “stovepipe”  of  another  type  than  the  typical 
closed  data  stream  discussed  above.  In  addition  to  the  restrictions  deriving  from  the  air- 
vs.  -groimd  data  distinctions,  the  two  operators  were  specifically  assigned  to  compile  and 
export  updated  reports  on  subjects  which  were  differentiated  along  yet  another  dimension 
~  Alpha  vs.  Bravo  sector.  Phrased  another  way,  each  operator  had  to  contend  with  the 
limitations  of  an  incoming  stovepipe  insulating  air  from  ground  data,  in  terms  of  collating 
data  from  his/her  own  and  the  other  data  stream.  In  addition  to  this  problem,  each 
operator  was  expected  to  compile  a  composite  data  product  from  his/her  best  synthesis  of 
these  two  data  streams  and  export  it  via  what  can  be  considered  an  outgoing  stovepipe 
insulating  Alpha  sector  from  Bravo  sector  results. 

Phrased  another  way,  each  of  the  BMC4I  demo  operators  had  to  contend  with  “crossed 
stovepipes.”  With  regard  to  incoming  data,  each  operator  had  to  concentrate  attention 
and  effort  with  respect  to  the  air  vs.  groimd  distinctions  governing  battlespace  component 
categorization  and  data  stream  allocations.  With  regard  to  compiling  an  informational 
product  (updates  to  the  Summary  Situation  Display),  each  operator  had  to  concentrate 
attention  and  effort  with  respect  to  the  Alpha  vs.  Bravo  distinction  governing  battlespace 
territorial  categorization  and  joint  force  allocations.  This  “switching”  necessarily  entailed 
doing  one  task  from  two  distinct  perspectives,  so  to  speak.  From  a  human  factors 
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viewpoint,  tiiis  would  lead  to  a  prediction  of  increased  cognitive  workload  for  each 
operator. 

d.  Mismatched  resource  allocations  among  data  stovepipes  and  other  restrictions  further 
complicate  operations 

As  noted  above,  each  operator  had  to  deal  with  two  distinct  types  of  “stovepiping”  --  air- 
vs.  -groimd  on  the  incoming  data  and  Alpha-vs.  -Bravo  on  the  outgoing  results.  Neither 
operator  could  accomplish  his/her  task  without  taking  the  time  to  (a)  distinguish  which 
data  coming  from  his/her  own  incoming  stovepipe  was  relevant  to  the  restrictions  of 
his/her  outgoing  stovepipe  and  (b)  determine  the  applicability  of  data  from  the  other’s 
incoming  stovepipe  to  his/her  owm  outgoing  stovepipe.  In  effect,  the  operators  were 
caught  in  a  procedural  crossfire  induced  by  the  figurative  “crossing”  or  intersection  of 
two  distinct  stovepipes  at  their  respective  stations. 

e.  The  affordances  of  groupware  applications  induce  restrictions  on  the  collaborators  ’ 
ability  to  work. 

In  the  case  of  the  BMC4I  demo,  the  groupware  (layer  6  in  Figure  in.3)  was  limited  to 
Microsoft  NetMeeting.  It  was  through  this  software  that  the  USAF  and  Army  operators 
communicated  (via  audio  /  video  conferencing)  and  jointly  assembled  their  Sector 
Summary  Display  information  (via  the  shared  whiteboard).  As  with  any  groupware 
application,  the  affordances  of  NetMeeting  affected  the  maimer  and  efficiency  of  the 
operators’  collaboration.  A  more  detailed  discussion  of  these  effects  (in  the  general 
context  of  COTS  groupware)  wall  be  given  in  Section  VI  (Groupware  Issues  Illustrated  in 
the  BMC4I  Demonstration). 


D.  Information  Displayed,  Shared,  and  Manipulated 

Each  of  the  two  Intergraph  operation  stations  was  equipped  with  dual  21"  monitors.  For 
the  purposes  of  the  demo,  we  developed  a  structured  screen  display  for  each  monitor.  On 
the  left-hand  monitor  was  a  mock-up  of  a  siunmary  situation  display.  On  the  right-hand 
monitor  was  a  mock-up  of  an  individual  operator  data  display.  These  basic  displays  are 
illustrated  below  in  Figures  6  and  8,  respectively.  In  the  following  sections,  each  of  these 
displays  will  be  introduced  and  its  component  subdisplays  explained.  The  subdisplay  / 
display  composition  for  each  mock-up  is  explained  in  Figures  7  and  9,  respectively. 

1.  The  Summary  Situation  Display  (Left  -  Hand  Monitor) 

The  Summary  Situation  Display  represents  the  shared  information  space  linking  each  of 
the  two  station  operators  and  their  command  center.  The  largest  element  of  this  screen 
display  was  the  Summary  Battlespace  Display  --  a  “God’s  Eye”  view  of  the  geographical 
context  for  the  simulated  operations.  Included  in  the  Summary  Battlespace  Display  were 
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symbolic  depictions  of  adversary  (red)  and  own  (blue)  forces,  phase  lines  (planned 
deployment  boundaries),  combat  air  patrol  (CAP)  circuits,  threat  zones  (areas  of 
adversary  SAM  coverage),  command  posts,  as  well  as  individual  /  grouped  platforms 
(specifically  armor  and  air). 


Figure  IIL6:  Summary  Situation  Display  (Left-hand  Monitor) 

On  the  right-hand  side  of  the  Summary  Situation  Display  were  two  smaller  windows  — 
each  designated  as  containing  composite  data  on  Alpha  and  Bravo  sectors,  respectively. 
These  windows  were  intended  to  represent  the  latest  available  status  on  operations  in  a 
given  sector.  As  such,  these  displays  had  to  present  fused,  filtered,  and  collated  data  as 
processed  by  fiie  two  operators  participating  in  the  simulated  mission. 
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Figure  III.7:  Components  of  the  Summary  Situation  Display 


2.  Individual  Operator  Display  (Right  -  Hand  Monitor) 

In  this  section,  the  composition  and  configuration  of  each  operator’s  individual  or  private 
on-screen  workspace  will  be  introduced.  Figure  HI.  8  illustrates  the  basic  layout  of  the 
Individual  Operator  Display,  and  Figure  in.9  annotates  this  layout  with  the  nomenclature 
to  be  used  in  this  section  to  introduce  and  explain  the  information  display’s  intended 
utility. 
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Figure  III.8:  Individual  Operator  Display  (Right-hand  Monitor) 


The  uppermost  half  of  the  hidividual  Operator  Display  eontains  a  row  of  three  windows 
essential  to  each  operator’s  simulated  task.  In  the  upper  left-hand  portion  of  the 
hidividual  Operator  Display  are  two  windows  —  each  depicting  incoming  battlespace 
event  data  for  the  joint  operation’s  air  and  ground  components,  respectively.  Each 
operator  had  the  ability  to  read  and  to  “copy”  the  data  from  each  of  these  windows.  In 
terms  of  inputs  to  their  tasks,  the  USAF  operator’s  primary  focus  was  the  Air  Event  Data 
Stream,  and  the  Army  operator’s  primary  focus  was  the  Ground  Event  Data  Stream.  Each 
operator  used  his  respective  Data  Stream  window  as  the  source  for  compiling  overall 
battlespace  status  for  his  designated  component  (air  vs.  ground). 


Because  each  of  the  operator  had  the  additional  task  of  compiling  and  forwarding  status 
information  by  sector  (i.e..  Alpha  vs.  Bravo),  both  operators  had  to  be  accorded  access  to 
both  Data  Stream  windows,  hi  compiling  the  required  sector-delimited  summaries,  the 
USAF  operator  had  to  augment  information  derived  from  the  Air  Event  Data  Stream  with 
items  derived  from  the  Ground  Event  Data  Stream,  and  vice  versa.  Even  in  the  limited 
context  of  this  demonstration,  this  clearly  illustrates  the  manner  in  which  organizational  / 
operational  factors  (the  task  assignments)  influence  the  provision  of  information  to 
operators  via  their  respective  “interfaces.” 


In  the  upper  right-hand  comer  is  a  window  containing  icons  for  a  variety  of  air  and 
ground  operational  elements  (e.g.,  aircraft  and  tanks).  Each  operator  had  the  ability  to 
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“copy  and  paste”  icons  from  this  Platform  Icon  Window  to  the  Summary  Battlespace 
Display  as  a  direct  manipulative  updating  of  asset  status  and  resource  allocation.  As 
noted  above  with  respect  to  the  Data  Stream  windows  and  compiling  status  information, 
each  operator  had  to  have  access  to  icons  for  both  (i.e.,  air  and  ground)  classes  of 
operational  elements  to  be  manipulated  on  the  Summary  Battlespace  Display.  Again, 
this  illustrates  the  manner  in  which  tasking  influences  the  aflFordances  of  each  operatdr’s 
“interface”  to  the  overall  BMC4I  system  of  systems. 


Figure  III.9:  Components  of  the  Individual  Operator  Display 


The  lower  half  of  the  Individual  Operator  Display  contains  four  windows  which  comprise 
the  groupware  component  of  the  BMC4I  demo.  These  windows  (all  realized  via 
Microsoft  NetMeeting)  provided  the  joint  work  and  commrmication  space  through  which 
the  operators  collaborated  in  the  simulated  task. 


In  the  lower  left-hand  comer  was  a  Shared  Whiteboard  window.  Each  operator  had  read, 
write,  and  modify  access  to  this  shared  resource.  It  was  in  the  Shared  Whiteboard 
window  that  the  operators  could  individually  add  and  jointly  review  their  updates  to  the 
Summary  Situation  Display.  In  the  extreme  lower  right-hand  comer  were  two  real-time 
video  windows,  in  which  the  feeds  from  each  operator’s  camera  appeared.  It  was  through 
these  video  windows  that  each  operator  could  see  the  other.  Above  the  video  display 
windows  were  the  Video  Conferencing  Controls  by  which  each  operator  accessed  and 
manipulated  the  communications  channel  between  the  two  workstations. 
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3.  Summaiy 

At  this  point,  we  can  now  summarize  the  BMC4I  setup  with  respect  to  the  CSTL  model 
of  collaborative  S3^tems  (cf.  Figure  ni.3).  In  Figure  HI.  1 0,  the  elements  of  the 
demonstration  are  organized  with  respect  to  the  relative  dependencies  discussed  above. 
The  square  boxes  are  used  to  illustrate  specific  imits  or  technologies  employed  in  the 
demonstration,  and  the  oval  boxes  depict  those  capabilities  or  affordances  supported  by 
the  units  /  technologies  listed  lower  in  the  figure.  These  two  types  of  elements  allow  us 
to  illustrate  the  mappings  for  both  the  apparatus  reviewed  in  section  IE  and  the 
operational  scenario  elements  reviewed  in  sections  IV  and  V.  The  use  of  three  vertical 
“background  bars”  permits  the  illustration  of  how  the  logical  /  fimctional  capacities  of  the 
BMC4I  demo  relate  to  the  three-workstation  platforms.  The  degree  to  which  the 
technology  and  capacities  “boxes”  span  the  three  vertical  bars  represents  the  degree  to 
which  the  factors  represented  by  those  boxes  span  the  three  workstations  employed  in  the 
demo. 


Summary  Situation  Display 
NetMeeting  (Application  Sharing) 

NetMeeting  H 

(Shared  Whiteboard)  H 

^Intei^per^rDialogue^^^^^^ 

NetMeeting  H 
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PowerPoint 
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Infonnational  /  Functional  Capability 


Figure  ffl.lO:  The  BMC4I  Demo  as  an  Instance  of  the  CSTL  Model 
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Figure  HI.  1 0  provides  a  coherent  framework  for  analyzing  the  topology  and 
interdependencies  among  the  elements  comprising  the  BMC4I  demo.  Most  importantly, 
it  provides  a  basis  for  linking  specific  technical  elements  (e.g.,  workstations,  software)  to 
specific  levels  of  collaborative  activity  afforded  by  a  given  subset  of  the  overall  technical 
system.  Finally,  Figure  HI.  10  constitutes  ah  illustrative  application  of  the  CSTL 
preliminary  model  (cf.  Figure  III.3),  thus  demonstrating  an  early  result  of  CSTL 
theoretical  work  to  date. 


E.  Groupware  Issues  Illustrated  in  the  BMC4I  Demonstration 

The  BMC4I  demo  setup  illustrates  multiple  functional  limitations  of  groupware  software 
available  on  the  market,  as  well  as  constraints  which  may  derive  from  the  specific  manner 
in  which  groupware  is  installed  and  information  streams  are  made  available  to  users. 
Brief  descriptions  of  some  of  these  limitations  are  given  in  the  remainder  of  this  section. 

1.  The  information  being  processed  was  not  dynamic 

There  was  no  dynamic  stream  of  new  data  arriving  in  any  of  the  Alpha,  Bravo,  Air,  or 
Ground  windows.  All  these  windows  contained  static,  pre-loaded  data  items  which  were 
scripted  in  advance.  In  an  actual  operational  context,  one  would  hope  that  these  could 
truly  be  data  “streams”,  with  new  items  arriving  on  a  continuous  basis.  With  respect  to 
the  detailed  demo  setup,  this  limitation  derived  from  the  fact  that  the  event  data  streams 
were  mocked  up  using  PowerPoint.  In  actual  applications,  the  same  restrictions  might 
appear  for  numerous  reasons. 

One  obvious  reason  would  be  that  the  presentation  of  the  data  /  information  streams  to 
end-users  occurs  through  intermediary  systems  --  i.e.,  the  original  streams  are  not  fed 
directly  to  the  terminal  workstations.  This  could  occur  where  the  BMC4I  SOS  is 
configured  in  such  a  way  that  these  streams  accrete  to  a  data  pool  (e.g.,  a  database 
management  system)  which  users  individually  access.  Intermittently  “static”  data  streams 
could  also  result  as  a  side  effect  of  application  locking  for  the  sake  of  turn  taking  among 
collaborators. 

To  use  the  specific  example  of  Microsoft  NetMeeting,  continuous  incoming  data  streams 
could  not  have  been  handled  in  any  case.  The  specific  mechanism  for  data  /  information 
sharing  (the  Shared  Whiteboard)  accepts  new  items  in  real  time  from  the  collaborators. 
There  is  no  mechanism  by  which  a  dynamically  updating  file  could  reside  in  the  Shared 
Whiteboard  per  se.  However,  the  application  sharing  capability  (as  was  done  to  afford 
joint  access  to  the  PowerPoint  files)  could  in  principle  provide  a  basis  for  joint  access  to 
some  remote  node  or  host  on  which  such  a  file  could  be  made  available. 


2.  The  operations  allowed  on  the  information  were  limited 
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Largely  as  a  result  of  the  static,  pre-loaded  nature  of  the  data  items  comprising  the 
majority  of  the  demonstration  displays,  operator  manipulations  were  effectively  limited  to 
“cut”,  “copy”,  and  “paste”  functions  among  windows.  There  were  3  specific  such 
manipulations  central  to  the  demonstration: 

1 .  transferral  of  items  from  the  incoming  Air  and  Ground  data  windows  to  the 
NetMeeting  shared  whiteboard  (for  joint  viewing  of  work  in  progress 

2.  transferral  of  items  from  the  incoming  Air  and  Ground  data  windows  to  the  Alpha  and 
Bravo  sector  summary  information  windows  on  the  Summary  Situation  Display  (cf 
Figures  6/ 7) 

3.  transferral  of  graphic  icons  from  the  Platform  Icons  window  (cf.  Figure  in.9)  to  the 
Summary  Battlespace  Display  window  (cf  Figure  HI.  7). 

Even  though  these  constraints  derived  fi’om  the  use  of  PowerPoint  mock-ups,  they  are  not 
limited  to  the  demonstration  scenario.  It  is  often  the  case  that  only  a  restricted  set  of 
manipulations  are  permitted  in  a  shared  information  space  —  something  which  derives 
from  the  fact  that  group  software  applications  must  commonly  invoke  application- 
specific  protocols  and  mechanisms  to  allow  common  access  and  jointly-accessible 
mampulations  on  the  shared  information.  Furthermore,  the  fact  that  there  are  few  (if  any) 
groupware  suites  providing  the  full  range  of  collaborative  functions  means  that  multiple 
groupware  applications  might  be  in  use  at  any  one  time.  In  such  a  case,  the  ability  to 
perform  even  the  modest  “copy  and  paste”  functions  mocked  up  in  this  demo  may  depend 
on  operating  system  capabilities  rather  than  the  capacities  of  the  groupware  applications 
perse. 


3.  Only  dyadic  interactions  were  represented  in  the  demo 

The  only  “players”  capable  of  interaction  and  collaboration  in  the  BMC4I  demo  were  the 
two  operators  seated  at  the  workstations.  The  simulated  command  center  (with  large- 
scale  display  of  the  summary  information)  did  not  have  any  operator  /  player  on  station, 
nor  did  it  provide  for  any  commimication  channel  back  to  either  operator.  As  such,  the 
BMG4I  arrangement  can  be  characterized  as  “two  players  and  one  observer.”  Another 
factor  limiting  interactivity  to  two  operators  relates  to  Microsoft  NetMeeting  (the  medium 
for  video  and  whiteboard  sharing).  Although  NetMeeting’s  shared  whiteboard  may  be 
accessed  by  an  arbitrary  number  of  collaborators,  the  application’s  videoconferencing 
capacity  is  limited  to  “point-to-point”  (i.e.,  between  two  given  stations).  Our  decision  to 
embed  video  conferencing  in  the  demo  therefore  entailed  a  maximiun  of  two  operators. 

This  latter  restriction  is  not  unique  to  NetMeeting.  Bandwidth,  processor  capacity,  and 
other  factors  make  such  point-to-point  constraints  common  among  networked 
conferencing  tools.  To  overcome  such  limitations  often  requires  escalating  the  cost  and 
complexity  of  the  network  infi'astructure  itself  (e.g,,  with  dedicated  video  routers)  or 
segregating  the  audio  /  video  conferencing  fimctions  onto  a  distinct  parallel  network 
infirastructure  (e.g.,  adding  a  dedicated  video  network  alongside  the  data  network). 
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4.  Behavioral  adjustments  were  mandated  by  NetMeeting’s  affordances 

The  technological  constraints  of  the  specific  groupware  (NetMeeting)  used  in  the  BMC4I 
demo  mandated  that  the  operators  act  within  the  “envelope”  of  actions  which  the  software 
could  handle.  For  example,  the  shared  whiteboard  facility  only  allowed  one  participant  to 
control  the  cursor  (and  hence  manipulations)  at  a  time.  This  induced  a  need  to  negotiate 
turn  taking  between  the  two  operators  --  a  process  which  consistently  introduced 
interruptions  or  delays  in  their  work.  Such  turn  taking  (and  its  potentially  problematical 
effects)  is  not  unique  to  NetMeeting.  It  has  been  a  long-standing  issue  in  CSCW  studies, 
and  it  remains  an  open  issue  for  research. 

5.  Summary 

The  specific  limitations  and  constraints  above  have  been  listed  for  the  sake  of  explaining 
some  particulars  of  why  the  BMC4I  demonstration  proceeded  as  it  did.  These  types  of 
limitations  were  not  unique  to  the  BMC4I  demo  setup,  and  must  be  faced  in  employing 
many  COTS  groupware  products.  As  such,  the  demo  substantively  illustrated  some  of  the 
key  issues  in  linking  and  supporting  collaborators  through  advanced  information 
technologies.  Research  in  (e.g.)  HCI  and  CSCW  has  long  addressed  such  topics.  One 
mi^t  well  wonder,  then,  about  the  extent  to  which  these  topics  are  still  open  areas  for 
constructive  research.  We  are  convinced  that  there  has  been  little  progress  to  date  along 
these  lines,  and  that  this  relative  deficiency  can  be  explained  with  respect  to  the  history  of 
providing  group  IT  support. 

Dining  the  last  3  decades,  the  capacity  for  collaboration  via  computer  networks  has 
evolved  in  such  a  way  as  to  mirror  the  topology  of  the  technologies’  deployment.  By  the 
end  of  the  1970’s,  one  could  afford  users  common  access  to  files  and  limited  joint 
interactivity,  providing  they  were  individually  accessing  a  common  hardware  platform 
(e.g.,  a  mainfi'ame)  via  relatively  “dumb”  workstations.  By  the  end  of  the  1980’s,  such 
abilities  were  afforded  in  the  emergent  desktop  /  LAN  deployment  paradigm  to  the  extent 
that  common  software  applications  could  operate  across  diverse  individual  workstations 
(e.g.,  PC’s)  connected  by  a  coherent  network  infi'astructure.  Although  end  users  had 
increased  the  scope  of  possible  manipulations  at  the  terminal  interface,  they  were  still 
constrained  by  issues  of  compatibility  across  their  respective  platforms.  As  We  approach 
the  end  of  the  1990’s,  we  are  seeing  the  emergence  of  viable  software  environments  (e.g., 
Java)  which  are  effectively  platform-independent.  At  this  initial  stage,  it  is  difficult  to 
ascertain  the  extent  to  which  current  platform-  and  protocol-oriented  features  will 
continue  to  constrain  collaborative  utility. 

The  point  is  that  the  types  of  collaborative  IT  capacities  built  into  the  BMC4I  demo  (file 
sharing,  shared  whiteboard,  video  conferencing)  is  not  really  new.  The  fact  of  the  matter 
is  that  there  has  been  little  (arguably  no)  new  class  of  functionality  made  available  to 
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team  workers  since  the  “old  days.”  Instead,  effort  has  been  directed  at  continuing  basic 
classes  of  collaborative  capacity  across  the  “divides”  among  successive  paradigms  of  IT 
deployment.  The  research  focus  most  relevant  to  CSTL’s  mission  (interfaces  for 
distributed  collaboration)  has  not  so  much  advanced  over  the  last  20  years  as  it  has  simply 
recycled  itself  in  response  to  shifting  implementation  modalities.  As  such,  the  work  we 
propose  to  pursue  is  as  yet  undone  even  after  all  these  years. 
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IV. 

PLANNED  ACTIVITIES 

1 

During  FY  98  we  plan  to  complete  an  experiment  that  will  test  the  value  to  team 
performance  provided  by  a  joint  functional  interface  that  explicitly  supports  expert-level 
team  and  individual  work  processes.  This  interface  will  be  contrasted  with  an  alternative 
one  intended  to  support  distributed  work  but  its  design  will  not  be  model  driven.  Good 
human  factors  practices  will  be  used  in  the  development  of  both  interface  concepts.  A 
suitable  team  task  containing  features  of  work  representative  of  an  ill-structured  battle 
management  work  domain  context  will  be  developed  and  used  in  this  experiment. 
Information  content  and  work  tools  will  be  identical  for  both  the  experimental  and  control 
interfaces.  Only  the  interfaces  themselves  will  be  different.  We  plan  to  use  three  person 
teams  for  this  experiment.  Adequate  training  will  be  provided  prior  to  formal  data 
collection  to  remove  novice  effects  from  work  processes.  The  experiment  will  be 
designed  such  that  team  performance  effects  due  to  training,  experience,  and  work- 
centered  interface  factors  can  be  separated.  Both  process  and  outcome  data  will  be 
collected  and  analyzed.  Results  will  be  used  to  refine  the  expertise  and  workspace 
models,  the  task  and  simulation  environment,  measurement  methods,  and  features  of  the 
interface  concept. 

The  research  will  be  conducted  in  the  Collaborative  Systems  Technology  laboratory 
within  the  Human  Effectiveness  Directorate  of  AFRL.  To  support  this  research, 
additional  lab  development  work  will  be  required  to  produce  a  dynamic,  interactive 
simulation  task,  as  opposed  to  the  canned  data  task  used  in  the  phase  one  BMC4I  demo  to 
gain  familiarity  with  and  knowledge  about  the  capabilities  and  limitations  of  Microsoft 
NetMeeting  as  a  collaborative  work  tool. 
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SUMMARY 


This  project  was  initiated  in  March  1997.  It  represents  a  new  research  activity  that  has 
been  established  to  provide  a  scientific  foundation  for  the  development  of  human- 
network  interfaces  that  facilitate  the  rapid  ormation,  maintenance,  and  mission 
performance  of  distributed  teams  operating  in  a  fluid,  fast-pace  military  context.  During 
this  initial  six  month  period,  we  have  made  significant  progress  in  four  areas:  (1) 
acquisition  of  knowledge  about  the  complexities  of  the  large-scale  battlefield 
management,  command,  control,  computers,  and  intelligence  (BMC4I)  work 
envirotunent;  (2)  acquisition  of  knowledge  about  the  emerging  information  technology 
infi'astructure  and  products  and  how  they  relate  to  distributed,  interactive,  and  shared 
work;  (3)  initial  development  of  a  theoretically  based  framework  for  use  both  to  structure 
an  empirical  research  program  on  the  interaction  between  collaborative  interfaces,  in  an 
IT  environment,  and  team  performance,  and  to  provide  a  basis  for  model-based 
predictions  of  team  performance;  and  (4)  initial  development  of  a  laboratory  to  support 
the  planned  empirical  research  aspects  of  the  project. 

The  work  that  confronts  teams  in  the  BMC4I  context  may  best  be  characterized  as 
involving  significant  real-time  probleming  solving.  For  distributed  teams,  problem 
solving  is  accomplished  through  an  information  technology  network  infrastructure.  The 
human  interfaces  to  this  infrastmcture,  therefore,  substantially  influence  problem-solving 
performance.  We  need  a  scientific  understanding  of  this  interaction.  Our  research 
framework  addesses  this  issue  from  the  perspective  of  what  is  known  about  experts  who 
solve  problems  in  complex  real-world  situations.  We  have  presented  a  sketch  of  the 
framework  which  includes  an  introductory  treatment  of  individual  ^d  team  expertise  and 
as  a  set  of  workspace  models  that  identifies  the  range  of  process  forms  experts  are 
believed  to  use.  The  model  serves  to  both  generate  hypotheses  about  expertise  and  how 
the  user  interfaces  (individiual  and  team  focus)  can  be  designed  to  facilitate  effective  and 
organized  group  work  performance. 

Based  on  our  analysis  we  have  established  a  conceptual  model  of  the  “user/team 
interface”  with  information  network  technology.  This  model  helps  to  clarify  the  various 
layers  and  fimctional  aspects  of  IT.  Furteher,  we  used  it  to  identify  a  joint  fimctional 
interface  to  support  teamwork  that  currently  is  not  well  formed  in  IT  environments.  One 
way  to  state  the  goal  of  this  research  program  is  to  provide  the  basis  for  deriving  design 
priniciples  to  be  used  in  the  design  of  the  joint  fimctional  interface. 

We  created  a  demonstration  of  a  distributed  collaborative  work  concept  in  the  BMC4I 
context.  The  main  purpose  for  the  demonstration  was  to  aid  us  in  checking  out  the 
interconnectivity  of  a  hetergenous  collection  of  hardware  and  software  that  define  our 
network  enviroment.  The  demonstration  also  provided  a  quick  look  at  team  issues  for  the 
BMC4I  concept. 
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In  a  very  short  period  of  time,  we  have  managed  to  put  into  place  the  basic  infrastructure 
needed  to  support  an  empirical  research  program  on  collaborative  work  in  an  IT 
environment.  During  the  next  year,  program  emphsis  will  shift  toward  accomplishing  an 
initial  set  of  experiments  designed  to  increase  our  understanding  of  expert  behavior  and 
to  test  hypotheses  about  the  relation  of  user  interface  features  and  expert  performance. 
We  will  also  be  developing  our  procedures  for  conducting  subsequent  large-scale 
experiments  that  can  adequately  differentiate  performance  effects  due  to  the  interface 
interaction  from  those  due  to  interactions  with  experience  and  training  factors. 
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